-- Logs begin at Wed 2022-10-05 19:42:19 UTC, end at Wed 2022-10-05 20:42:49 UTC. -- Oct 05 20:07:29.347211 dv-cinder-801522-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 5 20:07:29 2022] *** Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: nodename: dv-cinder-801522-12-iscsi Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: machine: x86_64 Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: clock source: unix Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: pcre jit disabled Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: detected number of CPU cores: 4 Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: current working directory: / Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: detected binary path: /usr/bin/uwsgi-core Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: your processes number limit is 47680 Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: your memory page size is 4096 bytes Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: detected max file descriptor number: 2048 Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: lock engine: pthread robust mutexes Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: thunder lock: enabled Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 05 20:07:29.358233 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 05 20:07:29.397446 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: Python main interpreter initialized at 0x55e7a31da9c0 Oct 05 20:07:29.397446 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: python threads support enabled Oct 05 20:07:29.397446 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: your server socket listen backlog is limited to 100 connections Oct 05 20:07:29.397446 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: your mercy for graceful operations on workers is 90 seconds Oct 05 20:07:29.397799 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: mapped 403077 bytes (393 KB) for 2 cores Oct 05 20:07:29.397879 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: *** Operational MODE: preforking *** Oct 05 20:07:29.398149 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: *** uWSGI is running in multiple interpreter mode *** Oct 05 20:07:29.398149 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: spawned uWSGI master process (pid: 81324) Oct 05 20:07:29.398334 dv-cinder-801522-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 05 20:07:29.398875 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: spawned uWSGI worker 1 (pid: 81337, cores: 1) Oct 05 20:07:29.400856 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: spawned uWSGI worker 2 (pid: 81338, cores: 1) Oct 05 20:07:29.400856 dv-cinder-801522-12-iscsi devstack@keystone.service[81324]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 05 20:07:33.636360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 20:07:33.636360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: from cryptography.utils import int_from_bytes Oct 05 20:07:33.637155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 20:07:33.637155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: from cryptography.utils import int_from_bytes Oct 05 20:07:33.637298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 20:07:33.637298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: from cryptography.utils import int_from_bytes Oct 05 20:07:33.637760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 20:07:33.637760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: from cryptography.utils import int_from_bytes Oct 05 20:07:33.686642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 05 20:07:33.686837 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 05 20:07:33.687022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] command line args: [] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 05 20:07:33.687112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 05 20:07:33.687278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 05 20:07:33.687363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 05 20:07:33.687706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ================================================================================ {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 05 20:07:33.687803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] command line args: [] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 05 20:07:33.687968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] admin_token = **** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.688052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 05 20:07:33.688160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ================================================================================ {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 05 20:07:33.688243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] config_dir = [] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.688387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.688470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] admin_token = **** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.688567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] config_source = [] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.688651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] config_dir = [] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.688747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] debug = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.688831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.688944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.689261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] config_source = [] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.689261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] debug = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.689464 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.689599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.689736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.689851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.689952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] list_limit = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] log_config_append = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] list_limit = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] log_config_append = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.690944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] log_dir = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] log_file = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] log_dir = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] log_file = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.691993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.692947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693032 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.693984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.694150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.694227 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.694403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.694681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.694759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.694875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] publish_errors = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.695080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.695163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.695308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.695385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] publish_errors = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.695486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.695575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.695730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.695849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.695955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.696049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.696208 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.696292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] standard_threads = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.696448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.696526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.696690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] standard_threads = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.696792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.696870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.696944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] use_journal = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] use_json = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] use_stderr = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] use_syslog = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] use_journal = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] use_json = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] use_stderr = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.697987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.698066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] use_syslog = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.698232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.698308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 20:07:33.698489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.698579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.698731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.698818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.698980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.699068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.699231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.699326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.699455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.699531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.699689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.699780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.699935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.700027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.700183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.700271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.700430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.700510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.700663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.700753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.700907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.700994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.701141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.701236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.701402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.701495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.701662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.701742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.701930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.702014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.702147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.702238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.702388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.702470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.702648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.702744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.702893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.703052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.703185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.703263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.703387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.703468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.703632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.703715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.703878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.703965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.704110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.704192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.704335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.704414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.704569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.704660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.704803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.704888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.705049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.705131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.705290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.705377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.705547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.705630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.705776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.705854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.706014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.706096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.706261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.706343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.706497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.706579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.706787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.706868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.707022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.707109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.707261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.707342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.707499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.707579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.707728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.707808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.707959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.708041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.708189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.708270 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.708420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.708502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.708753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.708814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.708876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.708933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.709106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.709188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.709339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.709425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.709589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.709682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.709819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.709903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.711064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.711311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.711518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.711713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.711818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.712093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.712321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.712510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.712699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.712809 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.713107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.713295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.713489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.713634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.713947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.714197 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.714395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.714592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.714774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.714997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.715193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.715386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.715489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.715737 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.715902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.716061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.716209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.716337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.716497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.716632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.716771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.716906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.717035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.717195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.717323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.717449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.717604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.717744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.717872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.718009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.718136 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.718262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.718387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.718563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.718693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.718819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.718953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.719081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.719214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.719348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.719475 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.719606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.719733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.719869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.720002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.720138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.720265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.720399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.720530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.720656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.720782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.720915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.721043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.721237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.721370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.721548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.721701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.721844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.721972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.722099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.722226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.722842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.722970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.723108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.723237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.723363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.723489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.723616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.723742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.723742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.723998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.724125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.724251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.724378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.724504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.724703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.724833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.724959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.725085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.725211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.725345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.725481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.725644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.725774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.725916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.726044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.726177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.726304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.726438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.726565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.726697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.726824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.726956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.727083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.727218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.727345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.727477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.727604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.727741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.727873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.728008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.728135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.728261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.728398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] auth.external = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.728526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.728678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.728814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.728942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.729079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.729207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.729335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] auth.external = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.729481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] auth.password = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.729662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.729793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] auth.token = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.729929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.730064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.730192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.730320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.730446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] auth.password = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.730574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.730710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] auth.token = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.730838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.730965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.731092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.731226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.731368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.731505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.731679 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.731814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.731941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] credential.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.732068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.732195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.732330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.732458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.732586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.732713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.732839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] credential.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.732966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.733111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.733239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.733367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.733493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.733645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.733774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.733915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.734050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.734177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.734310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.734439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.734571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.734708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.734838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.734975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.735109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.735236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.735364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.735498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.735628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.735755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.735884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.736016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.736150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] federation.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.736150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.736407 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.736536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.736674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.736801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.736929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.737064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.737192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] federation.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.737326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.737454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.737641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.737782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.737912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.738657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.738787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.738914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.739042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.739188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.739318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.739446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.739581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.739708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.739835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.739974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.740110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.740238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.740364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.740491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.740618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.740820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.740961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.741090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.741217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.741344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.741470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.741638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.741811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.741945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.742073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.742200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.742326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.742461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.742589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.742715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.742852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.742979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.743106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.743241 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.743369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.743496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.743622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.743748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.743875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.744009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.744137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.744265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.744392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.744533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.744661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.744798 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.744934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.745062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.745189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.745317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.745444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.745603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.745736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.745736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.745976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.746103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.746235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.746383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.746512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.746639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.746766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.746892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.747025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.747160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.747287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.747414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.747541 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.747667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.747799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.747934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.748072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.748199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.748325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.748452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.748578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.748712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.748840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.748973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.749100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.749233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.749369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.749497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.749693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.749821 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.749959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.750087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.750225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.750354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.750481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.750609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.750736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.750863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.750997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.750997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.751239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.751239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.751479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.751607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.751796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.751933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.752060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.752187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.752315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.752442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.752577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.752707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.752835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.752962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.753090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.753217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.753353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.753482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.753666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.753807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.753937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.754064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.754199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.754328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.754456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.754583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.754712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.754840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.754975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.755105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.755232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.755369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.755503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.755631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.755768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.755897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.755897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.756138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.756266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.756399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.756536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.756665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.756793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.756920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.757047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.757184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.757184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.757496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.757675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.757803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.757932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.758059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.758198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.758328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.758328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.758568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.758695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.758822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.758968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.759098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.759225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.759352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.759479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.759606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.759742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.759870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.759999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.760126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.760254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.760380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.760517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.760646 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.760784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.760912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.761040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.761167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.761302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.761430 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.761573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.761707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.761882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.762016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.762157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.762286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.762452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.762586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.762723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.762851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.762988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] resource.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.763118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.763245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.763372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.763500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.763627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.763763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.763892 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.764019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.764146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.764273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.764410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.764546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.764675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] resource.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.764802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.764928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.765056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.765183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.765318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.765446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.765641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] role.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.765770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.765897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] role.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.766103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.766235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.766373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.766501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.766628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.766755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.766898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.767035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.767169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.767304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.767438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] role.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.767571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.767713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] role.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.767848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.767983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.768116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.768259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.768395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.768537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.768673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.768807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.768941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.769074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.769739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.769840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.769917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.769989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770561 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770921 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.770991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.771917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772407 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] token.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.772980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.773932 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] token.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.774990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.connection = **** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775577 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.775932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776514 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.connection = **** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.776940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777220 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777368 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.777890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81338) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 05 20:07:33.778157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 20:07:33.778693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81337) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 05 20:07:33.802637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.802875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.803094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.803885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.804124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.804355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.811437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.812980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.840005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.840234 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.841429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.841691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.894360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.894969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.943204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.943422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.944547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:33.944787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.098456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.098706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.098863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.098944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.099030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.099107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.099181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.099305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.099388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.099516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.099607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.099733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.122446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.122651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.122735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.122889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.122972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.123111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.123194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.123324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.123414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.123535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.123616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.123745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.123827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.123957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.134906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.135247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.147933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.148146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.148379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.148485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.148604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.148713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.148820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.148945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.149035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.149171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.149252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81338) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.149390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.149637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.149855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81337) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 20:07:34.166313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.166522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 05 20:07:34.166861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.166942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.167059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 05 20:07:34.167155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 05 20:07:34.167330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.167460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.167592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.167673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 05 20:07:34.167846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.167929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.168099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.168179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.168433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.168695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.177655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.177937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.178207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.178398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.178481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.178684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.178959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.179245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.185336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.186233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.188482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.188766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.189047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.189327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.189408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.189637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.189735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.189902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.189984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.190168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.190249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.190437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.190519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.190703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.190784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.190971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.191567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.191768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.191849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.192085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.192363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.192638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.214644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.215735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.216110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.216377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.217211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.217500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.220987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.221275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.221579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.222191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.222511 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.222801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.227920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.228206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.229119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.229411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.233740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.234038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.234324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.234608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.234887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.235040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.235166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.235342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.235424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.235638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.235727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.235933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.236217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.236498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.236775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.237052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.250917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.251194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.252377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.252663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.255530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.256988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.258707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.259004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.260189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.260490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.263089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.264636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.266142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.267791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.269852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.270155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.270438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.270717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.270992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.271268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.271639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.271946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.272238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.272526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.272807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.273088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.286801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.287087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.287355 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.287616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.288789 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.289083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.289357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.289649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.294413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.295950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.296457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.297467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.297767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.298035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.299585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.299867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.300446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.300868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.301154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.301438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.301852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.302614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.302905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.303194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.303484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.303886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.313051 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.313344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.313647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.313928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.314305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.314591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.314878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.316590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.316883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.317166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.317447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.317855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.318143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.318438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.330566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.334016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.334329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.336970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.337235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.338433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.339877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.340148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.340232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.340549 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.341664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.343130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.343430 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.343833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.346837 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.348334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.350253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.351205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.351598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.351729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.351883 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.352161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.354611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.355015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.355306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.355592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.359853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.360143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.360422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.363402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.363709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.363995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.367274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81338) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.367558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.367835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.368106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.368378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.368646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.368924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.369198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.369464 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.369762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.370124 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.370388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.370662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.370855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81337) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 20:07:34.370949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81338) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.371142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.371437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.371720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.371999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.372277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.372555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.372838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.373121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.373398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.373812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.374086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.374357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.374624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81337) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 20:07:34.404979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e7a31da9c0 pid: 81338 (default app) Oct 05 20:07:34.408237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929e47d-3dd8-4839-a861-f489cd5b424b None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:34.408520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929e47d-3dd8-4839-a861-f489cd5b424b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:34.408789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929e47d-3dd8-4839-a861-f489cd5b424b None None] PATH_INFO: `/v3` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:34.409388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e7a31da9c0 pid: 81337 (default app) Oct 05 20:07:34.413440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Wed Oct 5 20:07:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:41.159734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf202c-90cb-40e1-b98f-4181b41b53a7 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:41.160209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf202c-90cb-40e1-b98f-4181b41b53a7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:41.160209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf202c-90cb-40e1-b98f-4181b41b53a7 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:41.164424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:41.169771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:41.170173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:41.170303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:41.240404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: INFO dbcounter [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] Registered counter for database keystone Oct 05 20:07:41.260796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG oslo_db.sqlalchemy.engines [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81338) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 20:07:41.267428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writer thread running {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 20:07:41.445727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG passlib.handlers.bcrypt [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81338) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 05 20:07:41.446049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG passlib.handlers.bcrypt [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81338) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 05 20:07:41.502974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:41.505132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:41.633064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 05 20:07:42.065143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf202c-90cb-40e1-b98f-4181b41b53a7 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:42.134397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: INFO dbcounter [None req-74bf202c-90cb-40e1-b98f-4181b41b53a7 None None] Registered counter for database keystone Oct 05 20:07:42.155538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG oslo_db.sqlalchemy.engines [None req-74bf202c-90cb-40e1-b98f-4181b41b53a7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81337) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 20:07:42.162127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writer thread running {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 20:07:42.279837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf202c-90cb-40e1-b98f-4181b41b53a7 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:42.284025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e6e96-7738-4ab7-9127-71946fa51b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:42.284675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e6e96-7738-4ab7-9127-71946fa51b9b admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:42.285463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e6e96-7738-4ab7-9127-71946fa51b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:42.285758 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e6e96-7738-4ab7-9127-71946fa51b9b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:42.293813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e6e96-7738-4ab7-9127-71946fa51b9b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:42.344419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e03e6e96-7738-4ab7-9127-71946fa51b9b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 05 20:07:42.344754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e6e96-7738-4ab7-9127-71946fa51b9b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:42.354629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Wed Oct 5 20:07:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:07:42.360636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:42.377069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b952ec9-e064-48aa-b371-160828dda34b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:42.381447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f359d6f-ec14-43de-ad42-014e3cd29a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:42.382555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f359d6f-ec14-43de-ad42-014e3cd29a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:42.382833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f359d6f-ec14-43de-ad42-014e3cd29a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:42.383099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f359d6f-ec14-43de-ad42-014e3cd29a53 admin admin] PATH_INFO: `/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:42.384710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f359d6f-ec14-43de-ad42-014e3cd29a53 admin admin] RBAC: Authorizing `identity:get_project(project_id=e6d76b1c203e49b8ae5f7f394a20a597)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:42.436736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3f359d6f-ec14-43de-ad42-014e3cd29a53 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 05 20:07:42.437077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f359d6f-ec14-43de-ad42-014e3cd29a53 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:42.442325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Wed Oct 5 20:07:42 2022] GET /identity/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:07:44.545663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8ebab-2301-4468-a465-1568161793d5 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:44.546195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8ebab-2301-4468-a465-1568161793d5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:44.546195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8ebab-2301-4468-a465-1568161793d5 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:44.550397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:07:44.555683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a1823-e8c0-4606-b09c-6d416ddbc5ee None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:44.556102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a1823-e8c0-4606-b09c-6d416ddbc5ee None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:44.556221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a1823-e8c0-4606-b09c-6d416ddbc5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:44.576055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-317a1823-e8c0-4606-b09c-6d416ddbc5ee None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:44.577595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-317a1823-e8c0-4606-b09c-6d416ddbc5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:44.585909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 05 20:07:44.853475 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8ebab-2301-4468-a465-1568161793d5 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:44.866382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8ebab-2301-4468-a465-1568161793d5 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:44.870691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887b742-9266-4f72-ad2d-e8bd9dab034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:44.871295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887b742-9266-4f72-ad2d-e8bd9dab034d admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:44.871574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887b742-9266-4f72-ad2d-e8bd9dab034d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:44.871844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887b742-9266-4f72-ad2d-e8bd9dab034d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:44.879644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887b742-9266-4f72-ad2d-e8bd9dab034d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:44.881406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887b742-9266-4f72-ad2d-e8bd9dab034d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:44.890001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Wed Oct 5 20:07:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:07:44.895401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a1823-e8c0-4606-b09c-6d416ddbc5ee None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:44.904388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a1823-e8c0-4606-b09c-6d416ddbc5ee None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:44.908688 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22752b46-12b0-4d3d-a571-b27584e4dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:44.909312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22752b46-12b0-4d3d-a571-b27584e4dd00 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:44.910165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22752b46-12b0-4d3d-a571-b27584e4dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:44.910473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22752b46-12b0-4d3d-a571-b27584e4dd00 admin admin] PATH_INFO: `/v3/users/c04d978bfed6492fb377cd274964302b` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:44.911786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22752b46-12b0-4d3d-a571-b27584e4dd00 admin admin] RBAC: Authorizing `identity:get_user(user_id=c04d978bfed6492fb377cd274964302b)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:44.913667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22752b46-12b0-4d3d-a571-b27584e4dd00 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:44.915170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Wed Oct 5 20:07:44 2022] GET /identity/v3/users/c04d978bfed6492fb377cd274964302b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:47.128470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3d560-1f19-46d9-b7aa-5e60b2181bd1 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:47.128957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3d560-1f19-46d9-b7aa-5e60b2181bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:47.128957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b3d560-1f19-46d9-b7aa-5e60b2181bd1 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:47.130290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:47.135408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2d48d-2bce-4b17-a977-88ad6f3dbd89 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:47.135783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2d48d-2bce-4b17-a977-88ad6f3dbd89 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:47.135962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2d48d-2bce-4b17-a977-88ad6f3dbd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:47.139759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f6f7f-dc4a-45b3-a16b-00249a83aa21 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:47.140040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f6f7f-dc4a-45b3-a16b-00249a83aa21 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:47.140312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f6f7f-dc4a-45b3-a16b-00249a83aa21 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:47.141500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:07:47.146863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb280c-325d-4606-9061-ba8101ab0170 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:47.147148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb280c-325d-4606-9061-ba8101ab0170 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:47.147424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb280c-325d-4606-9061-ba8101ab0170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:47.154396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7bf2d48d-2bce-4b17-a977-88ad6f3dbd89 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:47.155611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7bf2d48d-2bce-4b17-a977-88ad6f3dbd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:47.190200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG passlib.handlers.bcrypt [None req-9fbb280c-325d-4606-9061-ba8101ab0170 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81337) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 05 20:07:47.190582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG passlib.handlers.bcrypt [None req-9fbb280c-325d-4606-9061-ba8101ab0170 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81337) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 05 20:07:47.233990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:07:47.241123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9fbb280c-325d-4606-9061-ba8101ab0170 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:47.242748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9fbb280c-325d-4606-9061-ba8101ab0170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:47.340670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:07:47.500639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2d48d-2bce-4b17-a977-88ad6f3dbd89 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:47.525402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:47.603205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb280c-325d-4606-9061-ba8101ab0170 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:47.604874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2d48d-2bce-4b17-a977-88ad6f3dbd89 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:47.637818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a4371-a50d-4e01-a2a3-6b286c02aaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:47.638501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a4371-a50d-4e01-a2a3-6b286c02aaad None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:47.638776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a4371-a50d-4e01-a2a3-6b286c02aaad None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:47.639042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a4371-a50d-4e01-a2a3-6b286c02aaad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:47.650329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a4371-a50d-4e01-a2a3-6b286c02aaad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-899a4371-a50d-4e01-a2a3-6b286c02aaad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:07:47.659000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:07:47.660637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:07:47.661774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:07:47.666392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a4371-a50d-4e01-a2a3-6b286c02aaad None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:47.705934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb280c-325d-4606-9061-ba8101ab0170 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:47.708826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:47.744952 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb16c18-88d6-4d4f-badb-e6ddfdee1246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:47.745848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb16c18-88d6-4d4f-badb-e6ddfdee1246 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:47.746133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb16c18-88d6-4d4f-badb-e6ddfdee1246 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:47.746410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb16c18-88d6-4d4f-badb-e6ddfdee1246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:47.749007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:47.766912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a4371-a50d-4e01-a2a3-6b286c02aaad None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:47.799150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1d8c9-8e88-4ae9-a0c0-7910b2f5e2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:47.799745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1d8c9-8e88-4ae9-a0c0-7910b2f5e2cb None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:47.800018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1d8c9-8e88-4ae9-a0c0-7910b2f5e2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:47.800826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1d8c9-8e88-4ae9-a0c0-7910b2f5e2cb None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:47.801478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1d8c9-8e88-4ae9-a0c0-7910b2f5e2cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:07:47.801803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1d8c9-8e88-4ae9-a0c0-7910b2f5e2cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:47.803188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1d8c9-8e88-4ae9-a0c0-7910b2f5e2cb None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:47.813608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb16c18-88d6-4d4f-badb-e6ddfdee1246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:47.814907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:07:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:07:47.815116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb16c18-88d6-4d4f-badb-e6ddfdee1246 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:47.820527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1d8c9-8e88-4ae9-a0c0-7910b2f5e2cb None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:47.917423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:07:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 20:07:47.920365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1d8c9-8e88-4ae9-a0c0-7910b2f5e2cb None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:47.922908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb16c18-88d6-4d4f-badb-e6ddfdee1246 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:47.926139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:47.954277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed592578-e258-4ed9-a1cc-75e3f7779229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:47.954899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed592578-e258-4ed9-a1cc-75e3f7779229 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:47.955177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed592578-e258-4ed9-a1cc-75e3f7779229 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:47.955447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed592578-e258-4ed9-a1cc-75e3f7779229 None admin] PATH_INFO: `/v3/users/c04d978bfed6492fb377cd274964302b` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:47.956637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed592578-e258-4ed9-a1cc-75e3f7779229 None admin] RBAC: Authorizing `identity:get_user(user_id=c04d978bfed6492fb377cd274964302b)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:47.958108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed592578-e258-4ed9-a1cc-75e3f7779229 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:47.959645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:07:47 2022] GET /identity/v3/users/c04d978bfed6492fb377cd274964302b => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:07:47.965319 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed592578-e258-4ed9-a1cc-75e3f7779229 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:47.979852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:48.021771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb16c18-88d6-4d4f-badb-e6ddfdee1246 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:48.054899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489a3d9-0e11-46e1-a1c5-0abe3b022019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:48.055747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489a3d9-0e11-46e1-a1c5-0abe3b022019 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:48.056029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489a3d9-0e11-46e1-a1c5-0abe3b022019 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:48.056305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489a3d9-0e11-46e1-a1c5-0abe3b022019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:48.064611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed592578-e258-4ed9-a1cc-75e3f7779229 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:48.075229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489a3d9-0e11-46e1-a1c5-0abe3b022019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:48.076687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489a3d9-0e11-46e1-a1c5-0abe3b022019 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-1489a3d9-0e11-46e1-a1c5-0abe3b022019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:07:48.094634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:07:48.095525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:07:48.096311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:07:48.096311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:07:48.096311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:07:48.096311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:07:48.097411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ceb87-b4f2-4928-8ded-661f8c9cc10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:48.098079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ceb87-b4f2-4928-8ded-661f8c9cc10d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:48.098357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ceb87-b4f2-4928-8ded-661f8c9cc10d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:48.098630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ceb87-b4f2-4928-8ded-661f8c9cc10d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:48.099741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ceb87-b4f2-4928-8ded-661f8c9cc10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:48.101287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ceb87-b4f2-4928-8ded-661f8c9cc10d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:48.101536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489a3d9-0e11-46e1-a1c5-0abe3b022019 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:48.102860 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:07:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:07:48.108487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ceb87-b4f2-4928-8ded-661f8c9cc10d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:48.157288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:48.188544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:48.203851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489a3d9-0e11-46e1-a1c5-0abe3b022019 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:48.208189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ceb87-b4f2-4928-8ded-661f8c9cc10d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:48.236551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e942a5-b005-447e-9847-6306c52e98d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:48.237157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e942a5-b005-447e-9847-6306c52e98d0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:48.237436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e942a5-b005-447e-9847-6306c52e98d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:48.238272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e942a5-b005-447e-9847-6306c52e98d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:48.238916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e942a5-b005-447e-9847-6306c52e98d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:07:48.239224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e942a5-b005-447e-9847-6306c52e98d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:48.240303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9520ba2-c39b-4757-8f84-49b933eeae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:48.240631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e942a5-b005-447e-9847-6306c52e98d0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:48.240897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9520ba2-c39b-4757-8f84-49b933eeae4e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:48.241173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9520ba2-c39b-4757-8f84-49b933eeae4e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:48.241444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9520ba2-c39b-4757-8f84-49b933eeae4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:48.242192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9520ba2-c39b-4757-8f84-49b933eeae4e None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.domain.id=default, user.id=c04d978bfed6492fb377cd274964302b) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:07:48.242494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9520ba2-c39b-4757-8f84-49b933eeae4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:48.243909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9520ba2-c39b-4757-8f84-49b933eeae4e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:48.250458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Wed Oct 5 20:07:48 2022] GET /identity/v3/role_assignments?user.id=c04d978bfed6492fb377cd274964302b&scope.domain.id=default&role.id=36331ec562ff42f883fc883363bc4c8d => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:07:48.258749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:07:50.840064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6f9e8-c0fd-445f-9817-471b3d778b47 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:50.841081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6f9e8-c0fd-445f-9817-471b3d778b47 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:50.841606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6f9e8-c0fd-445f-9817-471b3d778b47 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:50.843402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:50.849135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e165a6d-d5a5-4aad-b3b8-92ae91a9fa71 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:50.849557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e165a6d-d5a5-4aad-b3b8-92ae91a9fa71 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:50.849747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e165a6d-d5a5-4aad-b3b8-92ae91a9fa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:50.869994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-2e165a6d-d5a5-4aad-b3b8-92ae91a9fa71 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:50.873432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2e165a6d-d5a5-4aad-b3b8-92ae91a9fa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:50.972960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:07:51.063716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf852b-8f55-4f2b-8fc8-495db32d4008 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:51.064335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf852b-8f55-4f2b-8fc8-495db32d4008 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:51.064828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf852b-8f55-4f2b-8fc8-495db32d4008 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:51.066401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:51.071942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ed644-d3c4-4927-9eeb-b50f714878aa None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:51.072237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ed644-d3c4-4927-9eeb-b50f714878aa None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:51.072507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ed644-d3c4-4927-9eeb-b50f714878aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:51.094047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f95ed644-d3c4-4927-9eeb-b50f714878aa None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:51.095159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f95ed644-d3c4-4927-9eeb-b50f714878aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:51.110551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf039f-d477-46b5-9fd6-b7dd05857617 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:51.110852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf039f-d477-46b5-9fd6-b7dd05857617 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:51.111096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf039f-d477-46b5-9fd6-b7dd05857617 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:51.112417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 14/25] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:51.117468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e63d6-a7c5-4813-b106-2a627365e57a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:51.117773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e63d6-a7c5-4813-b106-2a627365e57a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:51.118047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e63d6-a7c5-4813-b106-2a627365e57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:51.144446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-357e63d6-a7c5-4813-b106-2a627365e57a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:51.145686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-357e63d6-a7c5-4813-b106-2a627365e57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:51.190112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:51.247255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 12/26] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:07:51.291800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 15/27] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:07:51.345767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7423b7-461d-4ccc-bb75-cc9b854f6838 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:51.346057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7423b7-461d-4ccc-bb75-cc9b854f6838 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:51.346252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7423b7-461d-4ccc-bb75-cc9b854f6838 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:51.348135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 13/28] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:51.353412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031e0e-f3e5-46eb-a893-cd7a3b9cc074 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:51.353743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031e0e-f3e5-46eb-a893-cd7a3b9cc074 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:51.354001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031e0e-f3e5-46eb-a893-cd7a3b9cc074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:51.378473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7b031e0e-f3e5-46eb-a893-cd7a3b9cc074 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:51.379659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7b031e0e-f3e5-46eb-a893-cd7a3b9cc074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:51.467354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 16/29] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:07:51.536396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7423b7-461d-4ccc-bb75-cc9b854f6838 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:51.586516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b031e0e-f3e5-46eb-a893-cd7a3b9cc074 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:51.593766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:51.664563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7423b7-461d-4ccc-bb75-cc9b854f6838 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:51.699956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818ed2e-8c28-4121-970a-b85a4efd5ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:51.700591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818ed2e-8c28-4121-970a-b85a4efd5ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:51.700867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818ed2e-8c28-4121-970a-b85a4efd5ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:51.701137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818ed2e-8c28-4121-970a-b85a4efd5ca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:51.712115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818ed2e-8c28-4121-970a-b85a4efd5ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:51.713620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818ed2e-8c28-4121-970a-b85a4efd5ca9 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-2818ed2e-8c28-4121-970a-b85a4efd5ca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:07:51.729494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:07:51.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:07:51.731174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:07:51.731174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:07:51.731174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:07:51.731174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 14/30] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:07:51.731917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818ed2e-8c28-4121-970a-b85a4efd5ca9 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:51.770516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:51.789345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b031e0e-f3e5-46eb-a893-cd7a3b9cc074 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:51.838155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818ed2e-8c28-4121-970a-b85a4efd5ca9 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:51.873793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8b9ac-c08e-49ab-8834-92b77ee65edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:51.874532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8b9ac-c08e-49ab-8834-92b77ee65edd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:51.874715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8b9ac-c08e-49ab-8834-92b77ee65edd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:51.874988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8b9ac-c08e-49ab-8834-92b77ee65edd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:51.883838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08e2bd-ebba-472f-b37d-852821778af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:51.885040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08e2bd-ebba-472f-b37d-852821778af7 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:51.885601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08e2bd-ebba-472f-b37d-852821778af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:51.886073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08e2bd-ebba-472f-b37d-852821778af7 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:51.886666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8b9ac-c08e-49ab-8834-92b77ee65edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:07:51.887159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08e2bd-ebba-472f-b37d-852821778af7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:51.888765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08e2bd-ebba-472f-b37d-852821778af7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-3dc8b9ac-c08e-49ab-8834-92b77ee65edd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:07:51.896792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:07:51.897636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 15/31] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:07:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:07:51.898378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8b9ac-c08e-49ab-8834-92b77ee65edd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:51.998414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:52.007673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8b9ac-c08e-49ab-8834-92b77ee65edd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:52.064259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d5317-97ca-4b84-bf39-ea5afbb407b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:52.064882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d5317-97ca-4b84-bf39-ea5afbb407b1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.065160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d5317-97ca-4b84-bf39-ea5afbb407b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.065432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d5317-97ca-4b84-bf39-ea5afbb407b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.066137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d5317-97ca-4b84-bf39-ea5afbb407b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:07:52.066440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d5317-97ca-4b84-bf39-ea5afbb407b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:52.068004 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d5317-97ca-4b84-bf39-ea5afbb407b1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:52.068424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 17/32] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 20:07:51 2022] POST /identity/v3/roles => generated 219 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:07:52.069789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08e2bd-ebba-472f-b37d-852821778af7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:52.080860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 16/33] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:52.082362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d5317-97ca-4b84-bf39-ea5afbb407b1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:52.193843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:52.271331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d5317-97ca-4b84-bf39-ea5afbb407b1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:52.285791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08e2bd-ebba-472f-b37d-852821778af7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:52.313728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3bb1e-f369-4ee0-b563-e271b06e65d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:52.314345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3bb1e-f369-4ee0-b563-e271b06e65d7 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.314625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3bb1e-f369-4ee0-b563-e271b06e65d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.314896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3bb1e-f369-4ee0-b563-e271b06e65d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.315567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3bb1e-f369-4ee0-b563-e271b06e65d7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:52.317488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3bb1e-f369-4ee0-b563-e271b06e65d7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:52.324506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102d84-8192-4a62-8323-9ab1ffb320ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:52.325129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102d84-8192-4a62-8323-9ab1ffb320ca None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.325407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102d84-8192-4a62-8323-9ab1ffb320ca None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.325778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a102d84-8192-4a62-8323-9ab1ffb320ca None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.326505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102d84-8192-4a62-8323-9ab1ffb320ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:07:52.326920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102d84-8192-4a62-8323-9ab1ffb320ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:52.329638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a102d84-8192-4a62-8323-9ab1ffb320ca None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:52.342700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 18/34] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:07:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:52.343615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102d84-8192-4a62-8323-9ab1ffb320ca None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:52.455986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a102d84-8192-4a62-8323-9ab1ffb320ca None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:52.457209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 17/35] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 20:07:52 2022] POST /identity/v3/roles => generated 225 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:07:52.463230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59c1a8-a194-405f-addd-c665aa43083a None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.463886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59c1a8-a194-405f-addd-c665aa43083a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.465112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59c1a8-a194-405f-addd-c665aa43083a None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.467380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 18/36] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:52.468460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59c1a8-a194-405f-addd-c665aa43083a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:52.485215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:52.495928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:52.507562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87a940-b669-4547-a8e7-430a782b92d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:52.508284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87a940-b669-4547-a8e7-430a782b92d4 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.508667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87a940-b669-4547-a8e7-430a782b92d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.508940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87a940-b669-4547-a8e7-430a782b92d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.509644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87a940-b669-4547-a8e7-430a782b92d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:52.511395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87a940-b669-4547-a8e7-430a782b92d4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:52.557628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 19/37] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:07:52 2022] POST /identity/v3/projects => generated 301 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:07:52.557628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b365b-4163-483e-a13f-aee84d9685ad None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.557628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b365b-4163-483e-a13f-aee84d9685ad None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.557628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b365b-4163-483e-a13f-aee84d9685ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.585024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-671b365b-4163-483e-a13f-aee84d9685ad None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:52.586909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-671b365b-4163-483e-a13f-aee84d9685ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:52.638176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59c1a8-a194-405f-addd-c665aa43083a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:52.680176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea3b1c-6791-46d7-96a2-9268968389dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:52.680884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea3b1c-6791-46d7-96a2-9268968389dc None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.681173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea3b1c-6791-46d7-96a2-9268968389dc None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.681452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea3b1c-6791-46d7-96a2-9268968389dc None admin] PATH_INFO: `/v3/users/c04d978bfed6492fb377cd274964302b` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.683217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea3b1c-6791-46d7-96a2-9268968389dc None admin] RBAC: Authorizing `identity:get_user(user_id=c04d978bfed6492fb377cd274964302b)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:52.685692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea3b1c-6791-46d7-96a2-9268968389dc None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:52.686671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 19/38] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:07:52 2022] GET /identity/v3/users/c04d978bfed6492fb377cd274964302b => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:52.692790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea3b1c-6791-46d7-96a2-9268968389dc None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:52.709261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 20/39] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:07:52.788772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:52.814922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea3b1c-6791-46d7-96a2-9268968389dc None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:52.850979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620682e-d09d-415f-8aee-101985f96d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:52.851619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620682e-d09d-415f-8aee-101985f96d05 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.851900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620682e-d09d-415f-8aee-101985f96d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.852171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620682e-d09d-415f-8aee-101985f96d05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.853384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6620682e-d09d-415f-8aee-101985f96d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:52.854876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6620682e-d09d-415f-8aee-101985f96d05 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:52.856755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 20/40] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:07:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:07:52.862288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b365b-4163-483e-a13f-aee84d9685ad None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:52.863686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b934cc-10b4-45a3-a755-49a2d28c32ec None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.863972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b934cc-10b4-45a3-a755-49a2d28c32ec None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.864248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b934cc-10b4-45a3-a755-49a2d28c32ec None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.865877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 21/41] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:52.872441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35751-902a-4b28-80e5-332d0fadf0b0 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:52.872730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35751-902a-4b28-80e5-332d0fadf0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:52.873011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35751-902a-4b28-80e5-332d0fadf0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:52.896397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9db35751-902a-4b28-80e5-332d0fadf0b0 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:52.898568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9db35751-902a-4b28-80e5-332d0fadf0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:52.966587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 22/42] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:07:52.976713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b365b-4163-483e-a13f-aee84d9685ad None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:52.987237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:53.014386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74077d-7837-4d3a-958c-e02c8e44f737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:53.015053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74077d-7837-4d3a-958c-e02c8e44f737 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:53.015384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74077d-7837-4d3a-958c-e02c8e44f737 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:53.015665 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74077d-7837-4d3a-958c-e02c8e44f737 None admin] PATH_INFO: `/v3/domains/default/users/c04d978bfed6492fb377cd274964302b/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:53.017599 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74077d-7837-4d3a-958c-e02c8e44f737 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c04d978bfed6492fb377cd274964302b, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:53.019074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74077d-7837-4d3a-958c-e02c8e44f737 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:53.021384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35751-902a-4b28-80e5-332d0fadf0b0 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:53.034112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 21/43] 10.222.0.23 () {58 vars in 1361 bytes} [Wed Oct 5 20:07:52 2022] PUT /identity/v3/domains/default/users/c04d978bfed6492fb377cd274964302b/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:07:53.104156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:53.134116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35751-902a-4b28-80e5-332d0fadf0b0 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:53.170628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90829cd9-9870-4a5b-bfb6-f5780aaef47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:53.171255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90829cd9-9870-4a5b-bfb6-f5780aaef47c None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:53.171539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90829cd9-9870-4a5b-bfb6-f5780aaef47c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:53.171813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90829cd9-9870-4a5b-bfb6-f5780aaef47c None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:53.172497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90829cd9-9870-4a5b-bfb6-f5780aaef47c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:53.173981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90829cd9-9870-4a5b-bfb6-f5780aaef47c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:53.191006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 23/44] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 20:07:53 2022] POST /identity/v3/roles => generated 223 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:07:53.239078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74077d-7837-4d3a-958c-e02c8e44f737 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:53.454606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74077d-7837-4d3a-958c-e02c8e44f737 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:53.487233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ab748-4144-4671-b86f-413789272cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:53.488407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab748-4144-4671-b86f-413789272cad None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:53.488686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab748-4144-4671-b86f-413789272cad None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:53.488959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ab748-4144-4671-b86f-413789272cad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:53.490084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ab748-4144-4671-b86f-413789272cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:53.491539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ab748-4144-4671-b86f-413789272cad None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:53.493103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 22/45] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:07:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:53.498608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90829cd9-9870-4a5b-bfb6-f5780aaef47c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:53.597214 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90829cd9-9870-4a5b-bfb6-f5780aaef47c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:53.630303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55530c96-902e-4591-a082-3e0c6756b821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:53.630911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55530c96-902e-4591-a082-3e0c6756b821 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:53.631187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55530c96-902e-4591-a082-3e0c6756b821 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:53.631457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55530c96-902e-4591-a082-3e0c6756b821 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:53.632107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55530c96-902e-4591-a082-3e0c6756b821 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:53.633566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55530c96-902e-4591-a082-3e0c6756b821 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:53.638300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:53.662100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 24/46] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:07:53 2022] POST /identity/v3/projects => generated 312 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:07:55.199782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2295b07-d0a6-4223-bdde-e69fab6b796d None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:55.200286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2295b07-d0a6-4223-bdde-e69fab6b796d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:55.200286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2295b07-d0a6-4223-bdde-e69fab6b796d None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:55.201591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 23/47] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:55.207484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853a54-ae52-48ab-88dc-c9359e3cb7cc None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:55.207878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853a54-ae52-48ab-88dc-c9359e3cb7cc None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:55.208047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853a54-ae52-48ab-88dc-c9359e3cb7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:55.225849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a2853a54-ae52-48ab-88dc-c9359e3cb7cc None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:55.227024 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a2853a54-ae52-48ab-88dc-c9359e3cb7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:55.291170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 25/48] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:07:55.557440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2295b07-d0a6-4223-bdde-e69fab6b796d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:55.594468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:55.660620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2295b07-d0a6-4223-bdde-e69fab6b796d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:55.693241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5262df0-9e36-4749-9c1b-1c894a455dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:55.693936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5262df0-9e36-4749-9c1b-1c894a455dae None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:55.694224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5262df0-9e36-4749-9c1b-1c894a455dae None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:55.694500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5262df0-9e36-4749-9c1b-1c894a455dae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:55.706975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5262df0-9e36-4749-9c1b-1c894a455dae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-f5262df0-9e36-4749-9c1b-1c894a455dae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:07:55.712030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:07:55.712937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:07:55.713682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 24/49] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:07:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:07:55.719376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853a54-ae52-48ab-88dc-c9359e3cb7cc None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:55.815906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853a54-ae52-48ab-88dc-c9359e3cb7cc None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:55.819207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951d50c-c743-4226-80b0-ae11b8afac17 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:55.819495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951d50c-c743-4226-80b0-ae11b8afac17 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:55.819772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951d50c-c743-4226-80b0-ae11b8afac17 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:55.821098 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 25/50] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:55.826063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b40943-7828-41d3-b16b-1db44246d338 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:55.826359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b40943-7828-41d3-b16b-1db44246d338 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:55.826639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b40943-7828-41d3-b16b-1db44246d338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:55.833917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:55.844974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-30b40943-7828-41d3-b16b-1db44246d338 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:55.846700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-30b40943-7828-41d3-b16b-1db44246d338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:55.853198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f71dcd-4549-40ff-9949-b1a7ca407d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:55.853841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f71dcd-4549-40ff-9949-b1a7ca407d28 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:55.854127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f71dcd-4549-40ff-9949-b1a7ca407d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:55.854402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f71dcd-4549-40ff-9949-b1a7ca407d28 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:55.855073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f71dcd-4549-40ff-9949-b1a7ca407d28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:07:55.855377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f71dcd-4549-40ff-9949-b1a7ca407d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:55.856747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f71dcd-4549-40ff-9949-b1a7ca407d28 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:55.868319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 26/51] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:07:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:55.874761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f71dcd-4549-40ff-9949-b1a7ca407d28 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:55.917259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 26/52] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:07:56.075846 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f71dcd-4549-40ff-9949-b1a7ca407d28 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:56.108854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bedfc3-a2cd-4403-b1c7-9ae92457e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:56.109475 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bedfc3-a2cd-4403-b1c7-9ae92457e764 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:56.109780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bedfc3-a2cd-4403-b1c7-9ae92457e764 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:56.110048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bedfc3-a2cd-4403-b1c7-9ae92457e764 None admin] PATH_INFO: `/v3/users/c04d978bfed6492fb377cd274964302b` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:56.111148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bedfc3-a2cd-4403-b1c7-9ae92457e764 None admin] RBAC: Authorizing `identity:get_user(user_id=c04d978bfed6492fb377cd274964302b)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:56.113116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bedfc3-a2cd-4403-b1c7-9ae92457e764 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:56.115099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 27/53] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:07:55 2022] GET /identity/v3/users/c04d978bfed6492fb377cd274964302b => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:56.120558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b40943-7828-41d3-b16b-1db44246d338 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:56.147811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:56.187195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bedfc3-a2cd-4403-b1c7-9ae92457e764 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:56.219230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b40943-7828-41d3-b16b-1db44246d338 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:56.234388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:56.254427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133855-2cf6-4399-9de2-b2691abde262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:56.255059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133855-2cf6-4399-9de2-b2691abde262 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:56.255343 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133855-2cf6-4399-9de2-b2691abde262 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:56.255620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb133855-2cf6-4399-9de2-b2691abde262 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:56.256719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb133855-2cf6-4399-9de2-b2691abde262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:56.258229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb133855-2cf6-4399-9de2-b2691abde262 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:56.259798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 27/54] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:07:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:56.265525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133855-2cf6-4399-9de2-b2691abde262 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:56.287085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bedfc3-a2cd-4403-b1c7-9ae92457e764 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:56.320301 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70705c11-4564-401c-b62a-b1f9efc4b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:56.320918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70705c11-4564-401c-b62a-b1f9efc4b1fb None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:56.321198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70705c11-4564-401c-b62a-b1f9efc4b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:56.321469 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70705c11-4564-401c-b62a-b1f9efc4b1fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:56.322575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70705c11-4564-401c-b62a-b1f9efc4b1fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:56.324005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70705c11-4564-401c-b62a-b1f9efc4b1fb None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:56.325718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 28/55] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:07:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:07:56.331079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70705c11-4564-401c-b62a-b1f9efc4b1fb None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:56.355540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:56.362352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb133855-2cf6-4399-9de2-b2691abde262 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:56.395461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d38ad-8b1b-4a8b-a4a2-010741e6a473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:56.396072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d38ad-8b1b-4a8b-a4a2-010741e6a473 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:56.396347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d38ad-8b1b-4a8b-a4a2-010741e6a473 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:56.396618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d38ad-8b1b-4a8b-a4a2-010741e6a473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:56.397331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d38ad-8b1b-4a8b-a4a2-010741e6a473 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.domain.id=default, user.id=c04d978bfed6492fb377cd274964302b) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:07:56.397671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d38ad-8b1b-4a8b-a4a2-010741e6a473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:56.399108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d38ad-8b1b-4a8b-a4a2-010741e6a473 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:56.404886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 28/56] 10.222.0.23 () {56 vars in 1322 bytes} [Wed Oct 5 20:07:56 2022] GET /identity/v3/role_assignments?user.id=c04d978bfed6492fb377cd274964302b&scope.domain.id=default&role.id=36331ec562ff42f883fc883363bc4c8d => generated 530 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:07:56.429939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70705c11-4564-401c-b62a-b1f9efc4b1fb None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:56.445636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:56.466694 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a1c4f-e2db-4790-af31-e7a43aa5a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:56.467328 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a1c4f-e2db-4790-af31-e7a43aa5a1a9 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:56.467609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a1c4f-e2db-4790-af31-e7a43aa5a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:56.467881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a1c4f-e2db-4790-af31-e7a43aa5a1a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:56.468548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a1c4f-e2db-4790-af31-e7a43aa5a1a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:56.470058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a1c4f-e2db-4790-af31-e7a43aa5a1a9 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:56.498498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 29/57] 10.222.0.23 () {60 vars in 1140 bytes} [Wed Oct 5 20:07:56 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:07:58.643921 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8d07-fef7-46ad-a0fe-da022d73a61f None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:58.644431 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8d07-fef7-46ad-a0fe-da022d73a61f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:58.644515 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb8d07-fef7-46ad-a0fe-da022d73a61f None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:58.645849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 29/58] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:07:58.650914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37143a78-6dba-4415-b45f-c5a8e6739a8d None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:58.651337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37143a78-6dba-4415-b45f-c5a8e6739a8d None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:58.651464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37143a78-6dba-4415-b45f-c5a8e6739a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:58.668895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-37143a78-6dba-4415-b45f-c5a8e6739a8d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:07:58.670113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-37143a78-6dba-4415-b45f-c5a8e6739a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:07:58.734182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 30/59] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:07:59.003840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb8d07-fef7-46ad-a0fe-da022d73a61f None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:59.102807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb8d07-fef7-46ad-a0fe-da022d73a61f None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:59.135118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298fd24-6406-43ec-9dd0-0dde94c2dee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:59.135735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298fd24-6406-43ec-9dd0-0dde94c2dee1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:59.136017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298fd24-6406-43ec-9dd0-0dde94c2dee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:59.136290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298fd24-6406-43ec-9dd0-0dde94c2dee1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:59.137388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298fd24-6406-43ec-9dd0-0dde94c2dee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:59.138851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298fd24-6406-43ec-9dd0-0dde94c2dee1 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:59.140383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 30/60] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:07:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:07:59.145796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37143a78-6dba-4415-b45f-c5a8e6739a8d None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:07:59.234784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:07:59.241423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37143a78-6dba-4415-b45f-c5a8e6739a8d None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:07:59.274343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a63be-6bb4-45d1-8e28-d04e6816342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:07:59.274885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a63be-6bb4-45d1-8e28-d04e6816342d None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:07:59.275164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a63be-6bb4-45d1-8e28-d04e6816342d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:07:59.275435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a63be-6bb4-45d1-8e28-d04e6816342d None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:07:59.276086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a63be-6bb4-45d1-8e28-d04e6816342d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:07:59.277549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a63be-6bb4-45d1-8e28-d04e6816342d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:07:59.304080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 31/61] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:07:59 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:08:02.601453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db065099-dbbb-4dd2-bc68-c92c66704bd6 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:02.604020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db065099-dbbb-4dd2-bc68-c92c66704bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:02.604020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db065099-dbbb-4dd2-bc68-c92c66704bd6 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:02.604743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 31/62] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:02.609792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701bd83-4a8b-475e-80b4-c4debabe882e None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:02.610588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701bd83-4a8b-475e-80b4-c4debabe882e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:02.611077 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701bd83-4a8b-475e-80b4-c4debabe882e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:02.636176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c701bd83-4a8b-475e-80b4-c4debabe882e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:02.637924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf384703-65c5-4594-a890-c1fcc570bc16 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:02.638417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf384703-65c5-4594-a890-c1fcc570bc16 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:02.638887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf384703-65c5-4594-a890-c1fcc570bc16 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:02.640404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 32/63] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:02.641883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c701bd83-4a8b-475e-80b4-c4debabe882e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:02.645528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05d4d3-2412-4628-af99-5b8f9352d0ee None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:02.646053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05d4d3-2412-4628-af99-5b8f9352d0ee None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:02.646543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05d4d3-2412-4628-af99-5b8f9352d0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:02.672625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1b05d4d3-2412-4628-af99-5b8f9352d0ee None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:02.674939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1b05d4d3-2412-4628-af99-5b8f9352d0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:02.700968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:02.754135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 32/64] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:02.758400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022554a-5dfa-4003-978b-3f055484723f None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:02.758894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022554a-5dfa-4003-978b-3f055484723f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:02.759393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022554a-5dfa-4003-978b-3f055484723f None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:02.761250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 33/65] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:02.767404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed9ec2-3883-4323-9745-f1d652d24824 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:02.773353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed9ec2-3883-4323-9745-f1d652d24824 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:02.773353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed9ec2-3883-4323-9745-f1d652d24824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:02.779924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 33/66] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:02.793209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-1bed9ec2-3883-4323-9745-f1d652d24824 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:02.794446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1bed9ec2-3883-4323-9745-f1d652d24824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:02.871069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 34/67] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:03.152986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05d4d3-2412-4628-af99-5b8f9352d0ee None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.183918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed9ec2-3883-4323-9745-f1d652d24824 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.219867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:03.292029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05d4d3-2412-4628-af99-5b8f9352d0ee None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:03.322327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed9ec2-3883-4323-9745-f1d652d24824 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:03.330967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fe178-589d-409e-a395-72e43c723d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:03.332131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fe178-589d-409e-a395-72e43c723d72 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.332414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fe178-589d-409e-a395-72e43c723d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.332700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fe178-589d-409e-a395-72e43c723d72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.343058 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fe178-589d-409e-a395-72e43c723d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-c88fe178-589d-409e-a395-72e43c723d72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:03.348573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:03.349497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 34/68] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:03.354042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fe178-589d-409e-a395-72e43c723d72 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.357882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc199d9-8206-4cda-84b8-e88d8b69a249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:03.358525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc199d9-8206-4cda-84b8-e88d8b69a249 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.358810 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc199d9-8206-4cda-84b8-e88d8b69a249 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.359084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc199d9-8206-4cda-84b8-e88d8b69a249 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.369318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc199d9-8206-4cda-84b8-e88d8b69a249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:03.369448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-7fc199d9-8206-4cda-84b8-e88d8b69a249 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:03.374155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:03.374962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:03.375692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 35/69] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:03.376845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa7c26-6f8d-47eb-8cc4-742174aad063 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.377125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa7c26-6f8d-47eb-8cc4-742174aad063 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.377409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa7c26-6f8d-47eb-8cc4-742174aad063 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.378700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 36/70] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:03.379717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd42c8-b6f7-479d-92b1-5a5d3205d27a None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.380093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd42c8-b6f7-479d-92b1-5a5d3205d27a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.380370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd42c8-b6f7-479d-92b1-5a5d3205d27a None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.381571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 37/71] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:03.386248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd42c8-b6f7-479d-92b1-5a5d3205d27a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.453980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fe178-589d-409e-a395-72e43c723d72 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:03.456212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:03.488145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deeb37f-cc6a-45d4-af3a-69d2597b9471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:03.488766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deeb37f-cc6a-45d4-af3a-69d2597b9471 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.489042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deeb37f-cc6a-45d4-af3a-69d2597b9471 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.489310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deeb37f-cc6a-45d4-af3a-69d2597b9471 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.491015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd42c8-b6f7-479d-92b1-5a5d3205d27a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:03.499047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deeb37f-cc6a-45d4-af3a-69d2597b9471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-8deeb37f-cc6a-45d4-af3a-69d2597b9471 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:03.504338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:03.505243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 35/72] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:03.506338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deeb37f-cc6a-45d4-af3a-69d2597b9471 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.527342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27904db9-b5c7-46db-8aac-03b5f708cec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:03.527945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27904db9-b5c7-46db-8aac-03b5f708cec2 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.528224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27904db9-b5c7-46db-8aac-03b5f708cec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.528494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27904db9-b5c7-46db-8aac-03b5f708cec2 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.529147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27904db9-b5c7-46db-8aac-03b5f708cec2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:03.529446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27904db9-b5c7-46db-8aac-03b5f708cec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:03.530857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27904db9-b5c7-46db-8aac-03b5f708cec2 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:03.538784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 38/73] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:03.539895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f37c-7856-4fc5-81ca-d10e3cd3a860 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.540179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f37c-7856-4fc5-81ca-d10e3cd3a860 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.540456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f37c-7856-4fc5-81ca-d10e3cd3a860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.557699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c295f37c-7856-4fc5-81ca-d10e3cd3a860 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:03.558919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c295f37c-7856-4fc5-81ca-d10e3cd3a860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:03.582126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:03.605220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deeb37f-cc6a-45d4-af3a-69d2597b9471 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:03.623987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 39/74] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:03.625207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d1238-92d0-429a-93d6-b566a1007ebb None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.625498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d1238-92d0-429a-93d6-b566a1007ebb None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.625799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d1238-92d0-429a-93d6-b566a1007ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.646777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882da37-b0b3-43dc-ba57-bbb1d71eb020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:03.647945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882da37-b0b3-43dc-ba57-bbb1d71eb020 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.648225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882da37-b0b3-43dc-ba57-bbb1d71eb020 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.648503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882da37-b0b3-43dc-ba57-bbb1d71eb020 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.649173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882da37-b0b3-43dc-ba57-bbb1d71eb020 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:03.649671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882da37-b0b3-43dc-ba57-bbb1d71eb020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:03.651055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882da37-b0b3-43dc-ba57-bbb1d71eb020 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:03.653400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4c0d1238-92d0-429a-93d6-b566a1007ebb None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:03.654599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4c0d1238-92d0-429a-93d6-b566a1007ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:03.664251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 36/75] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:03.664251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882da37-b0b3-43dc-ba57-bbb1d71eb020 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.719986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:03.728781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 40/76] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:03.730487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d1238-92d0-429a-93d6-b566a1007ebb None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.767716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882da37-b0b3-43dc-ba57-bbb1d71eb020 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:03.808141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba5da2-cb21-47a2-8317-c9bf2ab77171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:03.808752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba5da2-cb21-47a2-8317-c9bf2ab77171 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.809030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba5da2-cb21-47a2-8317-c9bf2ab77171 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.809302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba5da2-cb21-47a2-8317-c9bf2ab77171 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.810096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba5da2-cb21-47a2-8317-c9bf2ab77171 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:03.810403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba5da2-cb21-47a2-8317-c9bf2ab77171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:03.812038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba5da2-cb21-47a2-8317-c9bf2ab77171 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:03.818485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:03.823101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 37/77] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:03.823836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba5da2-cb21-47a2-8317-c9bf2ab77171 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.866619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d1238-92d0-429a-93d6-b566a1007ebb None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:03.910122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848e0fb-2e23-4c33-81bf-5bc40768cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:03.911192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848e0fb-2e23-4c33-81bf-5bc40768cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.911192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848e0fb-2e23-4c33-81bf-5bc40768cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.911354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848e0fb-2e23-4c33-81bf-5bc40768cfc6 None admin] PATH_INFO: `/v3/users/cf6ad7faaf3247d5ba5962434f804797` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.927505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848e0fb-2e23-4c33-81bf-5bc40768cfc6 None admin] RBAC: Authorizing `identity:get_user(user_id=cf6ad7faaf3247d5ba5962434f804797)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:03.928959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8848e0fb-2e23-4c33-81bf-5bc40768cfc6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:03.935032 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 41/78] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/users/cf6ad7faaf3247d5ba5962434f804797 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:03.935032 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848e0fb-2e23-4c33-81bf-5bc40768cfc6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.947276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba5da2-cb21-47a2-8317-c9bf2ab77171 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:03.981185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def6a62-0009-448d-81cf-3aecc889d634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:03.982070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def6a62-0009-448d-81cf-3aecc889d634 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:03.982070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def6a62-0009-448d-81cf-3aecc889d634 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:03.982324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def6a62-0009-448d-81cf-3aecc889d634 None admin] PATH_INFO: `/v3/users/c04d978bfed6492fb377cd274964302b` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:03.983453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def6a62-0009-448d-81cf-3aecc889d634 None admin] RBAC: Authorizing `identity:get_user(user_id=c04d978bfed6492fb377cd274964302b)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:03.984916 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def6a62-0009-448d-81cf-3aecc889d634 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:03.986527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 38/79] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/users/c04d978bfed6492fb377cd274964302b => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:03.987297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def6a62-0009-448d-81cf-3aecc889d634 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:03.993330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.039381 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848e0fb-2e23-4c33-81bf-5bc40768cfc6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.057322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.074436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174854a0-24a0-467a-978e-f4259d4e1918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.075059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174854a0-24a0-467a-978e-f4259d4e1918 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.075338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174854a0-24a0-467a-978e-f4259d4e1918 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.075606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174854a0-24a0-467a-978e-f4259d4e1918 None admin] PATH_INFO: `/v3/users/cf6ad7faaf3247d5ba5962434f804797` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.076711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174854a0-24a0-467a-978e-f4259d4e1918 None admin] RBAC: Authorizing `identity:get_user(user_id=cf6ad7faaf3247d5ba5962434f804797)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:04.078174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174854a0-24a0-467a-978e-f4259d4e1918 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:04.079732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 42/80] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/users/cf6ad7faaf3247d5ba5962434f804797 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:04.080481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174854a0-24a0-467a-978e-f4259d4e1918 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:04.096194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def6a62-0009-448d-81cf-3aecc889d634 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.129275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfb949-af93-4ed8-88f0-c5ea8544fa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.129920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfb949-af93-4ed8-88f0-c5ea8544fa44 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.130199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfb949-af93-4ed8-88f0-c5ea8544fa44 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.130469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfb949-af93-4ed8-88f0-c5ea8544fa44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.140418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfb949-af93-4ed8-88f0-c5ea8544fa44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-7bbfb949-af93-4ed8-88f0-c5ea8544fa44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:04.146399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:04.147412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:04.148383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 39/81] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:04.150263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfb949-af93-4ed8-88f0-c5ea8544fa44 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:04.179868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174854a0-24a0-467a-978e-f4259d4e1918 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.200183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.215057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5730706-61e1-4154-8aa7-9aa7e8faa576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.215671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5730706-61e1-4154-8aa7-9aa7e8faa576 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.215954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5730706-61e1-4154-8aa7-9aa7e8faa576 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.216231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5730706-61e1-4154-8aa7-9aa7e8faa576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.248039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfb949-af93-4ed8-88f0-c5ea8544fa44 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.273086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.281220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e3069-59c8-4f12-8271-8e8f76bc4e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.281846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e3069-59c8-4f12-8271-8e8f76bc4e2b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.282131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e3069-59c8-4f12-8271-8e8f76bc4e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.282407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e3069-59c8-4f12-8271-8e8f76bc4e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.283144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5730706-61e1-4154-8aa7-9aa7e8faa576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:04.285114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5730706-61e1-4154-8aa7-9aa7e8faa576 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:04.348109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e3069-59c8-4f12-8271-8e8f76bc4e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:04.349584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e3069-59c8-4f12-8271-8e8f76bc4e2b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:04.381253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 43/82] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:04.382116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5730706-61e1-4154-8aa7-9aa7e8faa576 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:04.404645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.446420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 40/83] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:04.447264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e3069-59c8-4f12-8271-8e8f76bc4e2b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:04.474736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.483983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5730706-61e1-4154-8aa7-9aa7e8faa576 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.517604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9130a2-8b0a-4c72-8aa5-fda9988eceb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.518168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9130a2-8b0a-4c72-8aa5-fda9988eceb7 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.518452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9130a2-8b0a-4c72-8aa5-fda9988eceb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.519277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9130a2-8b0a-4c72-8aa5-fda9988eceb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.520406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9130a2-8b0a-4c72-8aa5-fda9988eceb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:04.521878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9130a2-8b0a-4c72-8aa5-fda9988eceb7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:04.523622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 44/84] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:04.524280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9130a2-8b0a-4c72-8aa5-fda9988eceb7 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:04.548037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e3069-59c8-4f12-8271-8e8f76bc4e2b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.580935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a10a6-714e-4cbb-9d2d-79b91efc1cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.582267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a10a6-714e-4cbb-9d2d-79b91efc1cad None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.582552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a10a6-714e-4cbb-9d2d-79b91efc1cad None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.582826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a10a6-714e-4cbb-9d2d-79b91efc1cad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.610139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.622281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9130a2-8b0a-4c72-8aa5-fda9988eceb7 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.648489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a10a6-714e-4cbb-9d2d-79b91efc1cad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:04.650007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a10a6-714e-4cbb-9d2d-79b91efc1cad None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:04.655659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6eee7-604a-4645-8ba6-831b6fc5a38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.656265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6eee7-604a-4645-8ba6-831b6fc5a38f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.656549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6eee7-604a-4645-8ba6-831b6fc5a38f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.656823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6eee7-604a-4645-8ba6-831b6fc5a38f None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.657483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6eee7-604a-4645-8ba6-831b6fc5a38f None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:04.657812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6eee7-604a-4645-8ba6-831b6fc5a38f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:04.659190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6eee7-604a-4645-8ba6-831b6fc5a38f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:04.667655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 45/85] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:04.670345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6eee7-604a-4645-8ba6-831b6fc5a38f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:04.678039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.749077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 41/86] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:04.750185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a10a6-714e-4cbb-9d2d-79b91efc1cad None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:04.769412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6eee7-604a-4645-8ba6-831b6fc5a38f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.802578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f2772-f715-4f18-ba36-736d618acbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.803300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f2772-f715-4f18-ba36-736d618acbbd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.803587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f2772-f715-4f18-ba36-736d618acbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.803863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f2772-f715-4f18-ba36-736d618acbbd None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.804980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f2772-f715-4f18-ba36-736d618acbbd None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:04.806484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f2772-f715-4f18-ba36-736d618acbbd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:04.808101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 46/87] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:04 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:04.808846 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f2772-f715-4f18-ba36-736d618acbbd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:04.826960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.851547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a10a6-714e-4cbb-9d2d-79b91efc1cad None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.880093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:04.887867 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b73dc-9814-443c-83b0-f5153be861ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.888502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b73dc-9814-443c-83b0-f5153be861ff None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.888783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b73dc-9814-443c-83b0-f5153be861ff None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.889057 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b73dc-9814-443c-83b0-f5153be861ff None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.890196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b73dc-9814-443c-83b0-f5153be861ff None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:04.891657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b73dc-9814-443c-83b0-f5153be861ff None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:04.898323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 42/88] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:04 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:04.898323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b73dc-9814-443c-83b0-f5153be861ff None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:04.914277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f2772-f715-4f18-ba36-736d618acbbd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:04.947752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fbb5e-da62-40f9-a218-a4cccb6fa489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:04.948367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fbb5e-da62-40f9-a218-a4cccb6fa489 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:04.948655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fbb5e-da62-40f9-a218-a4cccb6fa489 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:04.948926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fbb5e-da62-40f9-a218-a4cccb6fa489 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:04.949608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fbb5e-da62-40f9-a218-a4cccb6fa489 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:04.951047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fbb5e-da62-40f9-a218-a4cccb6fa489 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:04.971424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 47/89] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:08:04 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:08:04.972924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fbb5e-da62-40f9-a218-a4cccb6fa489 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:05.004665 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b73dc-9814-443c-83b0-f5153be861ff None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:05.042651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c760262-b85a-403b-960d-51b8d5c0dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:05.043215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c760262-b85a-403b-960d-51b8d5c0dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:05.043495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c760262-b85a-403b-960d-51b8d5c0dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:05.043767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c760262-b85a-403b-960d-51b8d5c0dcb4 None admin] PATH_INFO: `/v3/users/cf6ad7faaf3247d5ba5962434f804797` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:05.045006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c760262-b85a-403b-960d-51b8d5c0dcb4 None admin] RBAC: Authorizing `identity:get_user(user_id=cf6ad7faaf3247d5ba5962434f804797)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:05.046532 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c760262-b85a-403b-960d-51b8d5c0dcb4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:05.048191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 43/90] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:04 2022] GET /identity/v3/users/cf6ad7faaf3247d5ba5962434f804797 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:05.048998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c760262-b85a-403b-960d-51b8d5c0dcb4 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:05.070568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:05.104976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fbb5e-da62-40f9-a218-a4cccb6fa489 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:05.120192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:05.150963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cc518-602f-4609-b633-b43c9e1ec4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:05.151316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cc518-602f-4609-b633-b43c9e1ec4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:05.151494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cc518-602f-4609-b633-b43c9e1ec4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:05.152392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cc518-602f-4609-b633-b43c9e1ec4b7 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:05.152896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cc518-602f-4609-b633-b43c9e1ec4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:05.154376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cc518-602f-4609-b633-b43c9e1ec4b7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:05.156006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 48/91] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:04 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:05.156772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cc518-602f-4609-b633-b43c9e1ec4b7 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:05.167250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c760262-b85a-403b-960d-51b8d5c0dcb4 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:05.205438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb3fa6-bee4-4f88-adbe-be502f729a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:05.206840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb3fa6-bee4-4f88-adbe-be502f729a32 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:05.206840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb3fa6-bee4-4f88-adbe-be502f729a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:05.207042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb3fa6-bee4-4f88-adbe-be502f729a32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:05.208031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb3fa6-bee4-4f88-adbe-be502f729a32 None admin] RBAC: Adding query filter params (role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=d78c4808482c4e74b6e8a3037b83de37, user.id=cf6ad7faaf3247d5ba5962434f804797) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:05.208339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb3fa6-bee4-4f88-adbe-be502f729a32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:05.209859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb3fa6-bee4-4f88-adbe-be502f729a32 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:05.216551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 44/92] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:05 2022] GET /identity/v3/role_assignments?user.id=cf6ad7faaf3247d5ba5962434f804797&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=84d6c85b41034cae925afeb8921381e8 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:05.217216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb3fa6-bee4-4f88-adbe-be502f729a32 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:05.292524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cc518-602f-4609-b633-b43c9e1ec4b7 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:05.339249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1917d8-3cdc-4aff-af7a-6fb6a1f04fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:05.339686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1917d8-3cdc-4aff-af7a-6fb6a1f04fed None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:05.339985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1917d8-3cdc-4aff-af7a-6fb6a1f04fed None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:05.340232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1917d8-3cdc-4aff-af7a-6fb6a1f04fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:05.340968 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1917d8-3cdc-4aff-af7a-6fb6a1f04fed None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=d78c4808482c4e74b6e8a3037b83de37, user.id=c04d978bfed6492fb377cd274964302b) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:05.341273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1917d8-3cdc-4aff-af7a-6fb6a1f04fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:05.343216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1917d8-3cdc-4aff-af7a-6fb6a1f04fed None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:05.355720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 49/93] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:05 2022] GET /identity/v3/role_assignments?user.id=c04d978bfed6492fb377cd274964302b&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=36331ec562ff42f883fc883363bc4c8d => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:05.356508 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1917d8-3cdc-4aff-af7a-6fb6a1f04fed None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:05.367154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:05.376493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb3fa6-bee4-4f88-adbe-be502f729a32 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:05.423785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:05.438805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc6212-7d82-4b6f-b718-ff96a4e989e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:05.440483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc6212-7d82-4b6f-b718-ff96a4e989e4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:05.441503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc6212-7d82-4b6f-b718-ff96a4e989e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:05.442002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc6212-7d82-4b6f-b718-ff96a4e989e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:05.493910 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1917d8-3cdc-4aff-af7a-6fb6a1f04fed None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:05.536500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b3cfc-6ddd-4b17-9e1c-fe557dc5dfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:05.537145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc6212-7d82-4b6f-b718-ff96a4e989e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:05.537253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b3cfc-6ddd-4b17-9e1c-fe557dc5dfdd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:05.537655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b3cfc-6ddd-4b17-9e1c-fe557dc5dfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:05.537992 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b3cfc-6ddd-4b17-9e1c-fe557dc5dfdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:05.538784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b3cfc-6ddd-4b17-9e1c-fe557dc5dfdd None admin] RBAC: Adding query filter params (role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=d78c4808482c4e74b6e8a3037b83de37, user.id=cf6ad7faaf3247d5ba5962434f804797) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:05.539157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b3cfc-6ddd-4b17-9e1c-fe557dc5dfdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:05.539760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc6212-7d82-4b6f-b718-ff96a4e989e4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:05.540722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b3cfc-6ddd-4b17-9e1c-fe557dc5dfdd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:05.547122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 50/94] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:05 2022] GET /identity/v3/role_assignments?user.id=cf6ad7faaf3247d5ba5962434f804797&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:05.677019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 45/95] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:05.681087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b3cfc-6ddd-4b17-9e1c-fe557dc5dfdd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:05.770762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:05.814199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b3cfc-6ddd-4b17-9e1c-fe557dc5dfdd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:05.858631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fb7a4-183e-457b-bb35-c0e0762671a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:05.859339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fb7a4-183e-457b-bb35-c0e0762671a0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:05.859624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fb7a4-183e-457b-bb35-c0e0762671a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:05.859896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fb7a4-183e-457b-bb35-c0e0762671a0 None admin] PATH_INFO: `/v3/projects/c94fd2abfa064be2b7a1d3a6af2bbb05` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:05.861033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fb7a4-183e-457b-bb35-c0e0762671a0 None admin] RBAC: Authorizing `identity:get_project(project_id=c94fd2abfa064be2b7a1d3a6af2bbb05)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:05.867486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fb7a4-183e-457b-bb35-c0e0762671a0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:05.867486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 51/96] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:05 2022] GET /identity/v3/projects/c94fd2abfa064be2b7a1d3a6af2bbb05 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:05.873762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc6212-7d82-4b6f-b718-ff96a4e989e4 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:05.904775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:05.986807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc6212-7d82-4b6f-b718-ff96a4e989e4 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:06.038011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d8ef1-a22a-4983-a0c7-9a3898cce6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:06.038631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d8ef1-a22a-4983-a0c7-9a3898cce6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:06.038913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d8ef1-a22a-4983-a0c7-9a3898cce6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:06.039188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d8ef1-a22a-4983-a0c7-9a3898cce6a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:06.039925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d8ef1-a22a-4983-a0c7-9a3898cce6a7 None admin] RBAC: Adding query filter params (role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=c94fd2abfa064be2b7a1d3a6af2bbb05, user.id=cf6ad7faaf3247d5ba5962434f804797) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:06.040241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d8ef1-a22a-4983-a0c7-9a3898cce6a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:06.041723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d8ef1-a22a-4983-a0c7-9a3898cce6a7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:06.053840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 46/97] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:05 2022] GET /identity/v3/role_assignments?user.id=cf6ad7faaf3247d5ba5962434f804797&scope.project.id=c94fd2abfa064be2b7a1d3a6af2bbb05&role.id=84d6c85b41034cae925afeb8921381e8 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:08.021338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c14c2-e7e5-48ba-9dd2-18c244a0f2f9 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:08.021864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c14c2-e7e5-48ba-9dd2-18c244a0f2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:08.021932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c14c2-e7e5-48ba-9dd2-18c244a0f2f9 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:08.023506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 52/98] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:08.035352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac130-5d7c-4668-bbca-d9f04d944aea None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:08.035791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac130-5d7c-4668-bbca-d9f04d944aea None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:08.035888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bac130-5d7c-4668-bbca-d9f04d944aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:08.063423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-19bac130-5d7c-4668-bbca-d9f04d944aea None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:08.064629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-19bac130-5d7c-4668-bbca-d9f04d944aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:08.163463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 47/99] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:08.318233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459646c-d77a-4ab2-abca-2a3a4ba48895 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:08.318872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459646c-d77a-4ab2-abca-2a3a4ba48895 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:08.319338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459646c-d77a-4ab2-abca-2a3a4ba48895 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:08.320852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 53/100] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:08.335510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae60f4-7e05-45e4-bf73-87532cf50ad9 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:08.336088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae60f4-7e05-45e4-bf73-87532cf50ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:08.336571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae60f4-7e05-45e4-bf73-87532cf50ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:08.359725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-34ae60f4-7e05-45e4-bf73-87532cf50ad9 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:08.361224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-34ae60f4-7e05-45e4-bf73-87532cf50ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:08.383229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:08.452056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 48/101] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:08.556700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0743e8-f1b3-4216-a2da-698e247f1225 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:08.557337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0743e8-f1b3-4216-a2da-698e247f1225 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:08.557822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0743e8-f1b3-4216-a2da-698e247f1225 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:08.559346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 54/102] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:08.564980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75569e-974a-4294-b95a-dae3260394d3 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:08.566086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75569e-974a-4294-b95a-dae3260394d3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:08.566574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75569e-974a-4294-b95a-dae3260394d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:08.590538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5a75569e-974a-4294-b95a-dae3260394d3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:08.592082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5a75569e-974a-4294-b95a-dae3260394d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:08.602288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0743e8-f1b3-4216-a2da-698e247f1225 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:08.710749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 49/103] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:08.732554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9fb87-9a00-48e6-8d70-e10283e9d2d6 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:08.733132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9fb87-9a00-48e6-8d70-e10283e9d2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:08.733653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9fb87-9a00-48e6-8d70-e10283e9d2d6 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:08.735177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 50/104] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:08:08.741922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ebe27-25ae-4c9e-82ee-40c986cd5837 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:08.742448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ebe27-25ae-4c9e-82ee-40c986cd5837 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:08.742917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ebe27-25ae-4c9e-82ee-40c986cd5837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:08.771333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-714ebe27-25ae-4c9e-82ee-40c986cd5837 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:08.772823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-714ebe27-25ae-4c9e-82ee-40c986cd5837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:08.819955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0743e8-f1b3-4216-a2da-698e247f1225 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:08.895846 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951107ad-7971-4fc1-8fb5-343f8dfcd403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:08.896493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951107ad-7971-4fc1-8fb5-343f8dfcd403 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:08.897211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951107ad-7971-4fc1-8fb5-343f8dfcd403 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:08.897488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951107ad-7971-4fc1-8fb5-343f8dfcd403 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:08.907533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 51/105] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:08.908639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ebe27-25ae-4c9e-82ee-40c986cd5837 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:08.913775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:08.929358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951107ad-7971-4fc1-8fb5-343f8dfcd403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-951107ad-7971-4fc1-8fb5-343f8dfcd403 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:08.934543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:08.935369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:08.936488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 55/106] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:08.941933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951107ad-7971-4fc1-8fb5-343f8dfcd403 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:08.948169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:09.107163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951107ad-7971-4fc1-8fb5-343f8dfcd403 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:09.109963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ebe27-25ae-4c9e-82ee-40c986cd5837 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:09.149915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa04a9-c66c-41b4-8b6d-9d9bb8cd3f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:09.150775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa04a9-c66c-41b4-8b6d-9d9bb8cd3f0e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.151191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa04a9-c66c-41b4-8b6d-9d9bb8cd3f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.151594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa04a9-c66c-41b4-8b6d-9d9bb8cd3f0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.151999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c107d-50da-4b17-b969-503eb3375df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:09.152620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c107d-50da-4b17-b969-503eb3375df5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.152902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c107d-50da-4b17-b969-503eb3375df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.153179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c107d-50da-4b17-b969-503eb3375df5 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.153461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa04a9-c66c-41b4-8b6d-9d9bb8cd3f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:09.153920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c107d-50da-4b17-b969-503eb3375df5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:09.154225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c107d-50da-4b17-b969-503eb3375df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:09.155378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa04a9-c66c-41b4-8b6d-9d9bb8cd3f0e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:09.155598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c107d-50da-4b17-b969-503eb3375df5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:09.162578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 52/107] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:09.163979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed383b7c-b865-42f8-8da5-d29ab51d6efe None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.164422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed383b7c-b865-42f8-8da5-d29ab51d6efe None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.164883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 56/108] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:09.165054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed383b7c-b865-42f8-8da5-d29ab51d6efe None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.166019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c107d-50da-4b17-b969-503eb3375df5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:09.166325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 53/109] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:09.167025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed383b7c-b865-42f8-8da5-d29ab51d6efe None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:09.236655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:09.264024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:09.282966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c107d-50da-4b17-b969-503eb3375df5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:09.284935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed383b7c-b865-42f8-8da5-d29ab51d6efe None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:09.321842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55097af-9cbf-4f1c-a9c1-f04ad21dc942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:09.322482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55097af-9cbf-4f1c-a9c1-f04ad21dc942 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.322784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55097af-9cbf-4f1c-a9c1-f04ad21dc942 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.323054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55097af-9cbf-4f1c-a9c1-f04ad21dc942 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.323884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e16e63-43a0-43dd-8fba-25539e5b5331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:09.324548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e16e63-43a0-43dd-8fba-25539e5b5331 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.324831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e16e63-43a0-43dd-8fba-25539e5b5331 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.325161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e16e63-43a0-43dd-8fba-25539e5b5331 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.325841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e16e63-43a0-43dd-8fba-25539e5b5331 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:09.327303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e16e63-43a0-43dd-8fba-25539e5b5331 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:09.334752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55097af-9cbf-4f1c-a9c1-f04ad21dc942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-e55097af-9cbf-4f1c-a9c1-f04ad21dc942 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:09.340883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:09.341836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:09.342617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 57/110] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:09.343262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55097af-9cbf-4f1c-a9c1-f04ad21dc942 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:09.369627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 54/111] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:08:09 2022] POST /identity/v3/projects => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:08:09.369627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a4a44-ee9d-4414-ae4f-34163a246be0 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.369627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a4a44-ee9d-4414-ae4f-34163a246be0 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.369627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a4a44-ee9d-4414-ae4f-34163a246be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.382652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-866a4a44-ee9d-4414-ae4f-34163a246be0 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:09.383974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-866a4a44-ee9d-4414-ae4f-34163a246be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:09.478416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55097af-9cbf-4f1c-a9c1-f04ad21dc942 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:09.481382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 55/112] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:09.482225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a4a44-ee9d-4414-ae4f-34163a246be0 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:09.533917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:09.554075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db4e18-dc6d-4985-8428-4f8683ef6007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:09.555050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db4e18-dc6d-4985-8428-4f8683ef6007 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.555330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db4e18-dc6d-4985-8428-4f8683ef6007 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.555608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db4e18-dc6d-4985-8428-4f8683ef6007 None admin] PATH_INFO: `/v3/users/cf6ad7faaf3247d5ba5962434f804797` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.556905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db4e18-dc6d-4985-8428-4f8683ef6007 None admin] RBAC: Authorizing `identity:get_user(user_id=cf6ad7faaf3247d5ba5962434f804797)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:09.558462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db4e18-dc6d-4985-8428-4f8683ef6007 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:09.560151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 58/113] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:09 2022] GET /identity/v3/users/cf6ad7faaf3247d5ba5962434f804797 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:09.560967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db4e18-dc6d-4985-8428-4f8683ef6007 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:09.586353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:09.634124 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a4a44-ee9d-4414-ae4f-34163a246be0 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:09.678523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e3644-d446-47bb-919b-74f9ee8430cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:09.679572 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e3644-d446-47bb-919b-74f9ee8430cc None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.680064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e3644-d446-47bb-919b-74f9ee8430cc None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.680523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e3644-d446-47bb-919b-74f9ee8430cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.693120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e3644-d446-47bb-919b-74f9ee8430cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:09.695444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db4e18-dc6d-4985-8428-4f8683ef6007 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-394e3644-d446-47bb-919b-74f9ee8430cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:09.699353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:09.700194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:09.700919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 56/114] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:09.701987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e3644-d446-47bb-919b-74f9ee8430cc None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:09.733188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ac1fc-d231-4c68-82b4-09c6b15dea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:09.733926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ac1fc-d231-4c68-82b4-09c6b15dea50 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.734209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ac1fc-d231-4c68-82b4-09c6b15dea50 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.734505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ac1fc-d231-4c68-82b4-09c6b15dea50 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.735284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ac1fc-d231-4c68-82b4-09c6b15dea50 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:09.735593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ac1fc-d231-4c68-82b4-09c6b15dea50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:09.737000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ac1fc-d231-4c68-82b4-09c6b15dea50 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:09.746537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 59/115] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:09.752494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3467399-dfda-420c-abfa-a5cd958b7eae None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.752915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3467399-dfda-420c-abfa-a5cd958b7eae None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.753202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3467399-dfda-420c-abfa-a5cd958b7eae None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.754946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 60/116] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:09.755726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3467399-dfda-420c-abfa-a5cd958b7eae None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:09.805117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e3644-d446-47bb-919b-74f9ee8430cc None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:09.812175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:09.828260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:09.842120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bc826-e52f-429b-b125-f8719344c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:09.842756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bc826-e52f-429b-b125-f8719344c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.843039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bc826-e52f-429b-b125-f8719344c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.843316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bc826-e52f-429b-b125-f8719344c1c4 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.844456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479bc826-e52f-429b-b125-f8719344c1c4 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:09.846366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479bc826-e52f-429b-b125-f8719344c1c4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:09.853766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 57/117] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:09 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:09.853766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bc826-e52f-429b-b125-f8719344c1c4 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:09.875364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3467399-dfda-420c-abfa-a5cd958b7eae None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:09.912346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa712900-8a7f-47a7-a74b-0af73120df04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:09.912877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa712900-8a7f-47a7-a74b-0af73120df04 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.913156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa712900-8a7f-47a7-a74b-0af73120df04 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.913427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa712900-8a7f-47a7-a74b-0af73120df04 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.914111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa712900-8a7f-47a7-a74b-0af73120df04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:09.914630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa712900-8a7f-47a7-a74b-0af73120df04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:09.915844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa712900-8a7f-47a7-a74b-0af73120df04 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:09.929125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 61/118] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:09.929125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c1baa-599e-4ef9-8e23-55b3b27c5fc4 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:09.929125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c1baa-599e-4ef9-8e23-55b3b27c5fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:09.929125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c1baa-599e-4ef9-8e23-55b3b27c5fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:09.945776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-376c1baa-599e-4ef9-8e23-55b3b27c5fc4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:09.946951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-376c1baa-599e-4ef9-8e23-55b3b27c5fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:09.970811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bc826-e52f-429b-b125-f8719344c1c4 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:10.005326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49e5ec-8684-4b0f-a072-d0e9879e959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:10.006212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49e5ec-8684-4b0f-a072-d0e9879e959f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:10.008223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49e5ec-8684-4b0f-a072-d0e9879e959f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:10.008223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49e5ec-8684-4b0f-a072-d0e9879e959f None admin] PATH_INFO: `/v3/users/c5ff4a05676249e282fa93d97fcd6eb9` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:10.029537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 62/119] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:10.029537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c1baa-599e-4ef9-8e23-55b3b27c5fc4 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:10.032567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49e5ec-8684-4b0f-a072-d0e9879e959f None admin] RBAC: Authorizing `identity:get_user(user_id=c5ff4a05676249e282fa93d97fcd6eb9)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:10.034794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49e5ec-8684-4b0f-a072-d0e9879e959f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:10.036630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 58/120] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:09 2022] GET /identity/v3/users/c5ff4a05676249e282fa93d97fcd6eb9 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:10.037430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49e5ec-8684-4b0f-a072-d0e9879e959f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:10.090360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:10.097922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:10.155521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49e5ec-8684-4b0f-a072-d0e9879e959f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:10.169809 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c1baa-599e-4ef9-8e23-55b3b27c5fc4 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:10.197924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf3de8-e9e0-47bd-a313-924b656dfda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:10.198594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf3de8-e9e0-47bd-a313-924b656dfda5 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:10.198872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf3de8-e9e0-47bd-a313-924b656dfda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:10.199143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf3de8-e9e0-47bd-a313-924b656dfda5 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37/users/cf6ad7faaf3247d5ba5962434f804797/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:10.201639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf3de8-e9e0-47bd-a313-924b656dfda5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d78c4808482c4e74b6e8a3037b83de37, user_id=cf6ad7faaf3247d5ba5962434f804797, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:10.203784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf3de8-e9e0-47bd-a313-924b656dfda5 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:10.215909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599bac2-46a1-47b5-9d46-8273bed73a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:10.217176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599bac2-46a1-47b5-9d46-8273bed73a24 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:10.218189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599bac2-46a1-47b5-9d46-8273bed73a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:10.218720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599bac2-46a1-47b5-9d46-8273bed73a24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:10.225590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 59/121] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:08:10 2022] PUT /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37/users/cf6ad7faaf3247d5ba5962434f804797/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:10.226783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf3de8-e9e0-47bd-a313-924b656dfda5 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:10.244933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599bac2-46a1-47b5-9d46-8273bed73a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-b599bac2-46a1-47b5-9d46-8273bed73a24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:10.255157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:10.256061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:10.256800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:10.260098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 63/122] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:08:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:10.263858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599bac2-46a1-47b5-9d46-8273bed73a24 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:10.403719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf3de8-e9e0-47bd-a313-924b656dfda5 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:10.410884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599bac2-46a1-47b5-9d46-8273bed73a24 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:10.412610 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:10.418183 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:10.459045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d917c8-9807-4807-8f55-7a04f991b641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:10.459730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d917c8-9807-4807-8f55-7a04f991b641 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:10.460016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d917c8-9807-4807-8f55-7a04f991b641 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:10.460287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d917c8-9807-4807-8f55-7a04f991b641 None admin] PATH_INFO: `/v3/users/cf6ad7faaf3247d5ba5962434f804797` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:10.461415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d917c8-9807-4807-8f55-7a04f991b641 None admin] RBAC: Authorizing `identity:get_user(user_id=cf6ad7faaf3247d5ba5962434f804797)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:10.462896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d917c8-9807-4807-8f55-7a04f991b641 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:10.464480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 60/123] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:10 2022] GET /identity/v3/users/cf6ad7faaf3247d5ba5962434f804797 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:10.464763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252a79f-a608-48db-a8b4-3faee55f5f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:10.465239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d917c8-9807-4807-8f55-7a04f991b641 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:10.465505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252a79f-a608-48db-a8b4-3faee55f5f3c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:10.465951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252a79f-a608-48db-a8b4-3faee55f5f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:10.466291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252a79f-a608-48db-a8b4-3faee55f5f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:10.551898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252a79f-a608-48db-a8b4-3faee55f5f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:10.554397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252a79f-a608-48db-a8b4-3faee55f5f3c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:10.594900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d917c8-9807-4807-8f55-7a04f991b641 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:10.646356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10008db6-f152-4969-b3b2-c7038a7c185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:10.647671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10008db6-f152-4969-b3b2-c7038a7c185b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:10.648125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10008db6-f152-4969-b3b2-c7038a7c185b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:10.648798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10008db6-f152-4969-b3b2-c7038a7c185b None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:10.649695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10008db6-f152-4969-b3b2-c7038a7c185b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:10.650401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10008db6-f152-4969-b3b2-c7038a7c185b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:10.652741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10008db6-f152-4969-b3b2-c7038a7c185b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:10.669427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 61/124] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:10.670888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10008db6-f152-4969-b3b2-c7038a7c185b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:10.677207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 64/125] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:10.682174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252a79f-a608-48db-a8b4-3faee55f5f3c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:10.684696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:10.723556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:10.795150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10008db6-f152-4969-b3b2-c7038a7c185b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:10.800065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252a79f-a608-48db-a8b4-3faee55f5f3c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:10.832336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83aae-e1b5-496b-aff8-558a1d1b9045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:10.832976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83aae-e1b5-496b-aff8-558a1d1b9045 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:10.833254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83aae-e1b5-496b-aff8-558a1d1b9045 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:10.833543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83aae-e1b5-496b-aff8-558a1d1b9045 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:10.839811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bdddd-17ce-42fb-9477-9b707a282fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:10.840954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bdddd-17ce-42fb-9477-9b707a282fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:10.841811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bdddd-17ce-42fb-9477-9b707a282fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:10.842281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bdddd-17ce-42fb-9477-9b707a282fc0 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:10.843868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bdddd-17ce-42fb-9477-9b707a282fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:10.844314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d83aae-e1b5-496b-aff8-558a1d1b9045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:10.845823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bdddd-17ce-42fb-9477-9b707a282fc0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:10.848226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 65/126] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:10 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:10.849307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bdddd-17ce-42fb-9477-9b707a282fc0 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-47d83aae-e1b5-496b-aff8-558a1d1b9045 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:10.850271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:10.851088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:10.851816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 62/127] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:10.852406 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83aae-e1b5-496b-aff8-558a1d1b9045 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:10.931390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:10.969652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:10.975266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bdddd-17ce-42fb-9477-9b707a282fc0 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:10.979418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83aae-e1b5-496b-aff8-558a1d1b9045 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:11.012481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27687d-517b-4f0f-b4e5-6c1301d0cb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:11.013136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27687d-517b-4f0f-b4e5-6c1301d0cb3d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.013417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27687d-517b-4f0f-b4e5-6c1301d0cb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.013732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27687d-517b-4f0f-b4e5-6c1301d0cb3d None admin] PATH_INFO: `/v3/users/c04d978bfed6492fb377cd274964302b` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.014906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27687d-517b-4f0f-b4e5-6c1301d0cb3d None admin] RBAC: Authorizing `identity:get_user(user_id=c04d978bfed6492fb377cd274964302b)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:11.016364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27687d-517b-4f0f-b4e5-6c1301d0cb3d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:11.018342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 66/128] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:10 2022] GET /identity/v3/users/c04d978bfed6492fb377cd274964302b => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:11.018937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27687d-517b-4f0f-b4e5-6c1301d0cb3d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:11.030283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f969c-eea3-41c8-941d-a2522e8d41a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:11.030929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f969c-eea3-41c8-941d-a2522e8d41a7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.031209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f969c-eea3-41c8-941d-a2522e8d41a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.031538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f969c-eea3-41c8-941d-a2522e8d41a7 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.032715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f969c-eea3-41c8-941d-a2522e8d41a7 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:11.034829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f969c-eea3-41c8-941d-a2522e8d41a7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:11.036384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 63/129] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:10 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:11.037401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f969c-eea3-41c8-941d-a2522e8d41a7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:11.142736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27687d-517b-4f0f-b4e5-6c1301d0cb3d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:11.146697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f969c-eea3-41c8-941d-a2522e8d41a7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:11.182321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973a2f-4837-49c8-b361-1b1b66f2620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:11.182954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973a2f-4837-49c8-b361-1b1b66f2620a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.183225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973a2f-4837-49c8-b361-1b1b66f2620a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.183506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973a2f-4837-49c8-b361-1b1b66f2620a None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.184338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973a2f-4837-49c8-b361-1b1b66f2620a None admin] RBAC: Adding query filter params (name=member) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:11.184647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973a2f-4837-49c8-b361-1b1b66f2620a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:11.185391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828716a-708c-486d-b098-26eff33338f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:11.186205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973a2f-4837-49c8-b361-1b1b66f2620a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:11.186877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828716a-708c-486d-b098-26eff33338f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.187345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828716a-708c-486d-b098-26eff33338f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.187921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828716a-708c-486d-b098-26eff33338f5 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37/users/cf6ad7faaf3247d5ba5962434f804797/roles/6a600852dc8442fdb2ffc3d663e274ff` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.190894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828716a-708c-486d-b098-26eff33338f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d78c4808482c4e74b6e8a3037b83de37, user_id=cf6ad7faaf3247d5ba5962434f804797, role_id=6a600852dc8442fdb2ffc3d663e274ff)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:11.192794 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828716a-708c-486d-b098-26eff33338f5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:11.196558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 64/130] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:11.197476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973a2f-4837-49c8-b361-1b1b66f2620a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:11.206684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:11.216308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 67/131] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:08:11 2022] PUT /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37/users/cf6ad7faaf3247d5ba5962434f804797/roles/6a600852dc8442fdb2ffc3d663e274ff => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:11.217663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828716a-708c-486d-b098-26eff33338f5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:11.238307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:11.401298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973a2f-4837-49c8-b361-1b1b66f2620a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:11.405983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828716a-708c-486d-b098-26eff33338f5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:11.455477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8709f-ce55-4fdb-80f8-e387b1dddcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:11.457106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8709f-ce55-4fdb-80f8-e387b1dddcbc None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.457487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9b14d-7e66-498a-ad24-1bb197893ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:11.457876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8709f-ce55-4fdb-80f8-e387b1dddcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.458128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b14d-7e66-498a-ad24-1bb197893ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.458409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b14d-7e66-498a-ad24-1bb197893ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.458683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b14d-7e66-498a-ad24-1bb197893ba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.458918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8709f-ce55-4fdb-80f8-e387b1dddcbc None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.459467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9b14d-7e66-498a-ad24-1bb197893ba5 None admin] RBAC: Adding query filter params (role.id=853e41e951d14277bb9e21a8861bdbf9, scope.project.id=d78c4808482c4e74b6e8a3037b83de37, user.id=c5ff4a05676249e282fa93d97fcd6eb9) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:11.459782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9b14d-7e66-498a-ad24-1bb197893ba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:11.460592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8709f-ce55-4fdb-80f8-e387b1dddcbc None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:11.461364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9b14d-7e66-498a-ad24-1bb197893ba5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:11.462447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8709f-ce55-4fdb-80f8-e387b1dddcbc None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:11.464583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 65/132] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:11 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:11.466694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8709f-ce55-4fdb-80f8-e387b1dddcbc None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:11.472645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 68/133] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:11 2022] GET /identity/v3/role_assignments?user.id=c5ff4a05676249e282fa93d97fcd6eb9&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=853e41e951d14277bb9e21a8861bdbf9 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:11.474034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9b14d-7e66-498a-ad24-1bb197893ba5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:11.611059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:11.616165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:11.634290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8709f-ce55-4fdb-80f8-e387b1dddcbc None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:11.656476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9b14d-7e66-498a-ad24-1bb197893ba5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:11.704480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b0322-115b-4458-af78-8487d94ea670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:11.705167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b0322-115b-4458-af78-8487d94ea670 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.705447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b0322-115b-4458-af78-8487d94ea670 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.705757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b0322-115b-4458-af78-8487d94ea670 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37/users/c04d978bfed6492fb377cd274964302b/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.709339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c67cc7-a9cd-456c-82a5-3deae4c4bbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:11.710026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c67cc7-a9cd-456c-82a5-3deae4c4bbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.710304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c67cc7-a9cd-456c-82a5-3deae4c4bbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.710574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c67cc7-a9cd-456c-82a5-3deae4c4bbc9 None admin] PATH_INFO: `/v3/users/cf6ad7faaf3247d5ba5962434f804797` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.712554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b0322-115b-4458-af78-8487d94ea670 None admin] RBAC: Authorizing `identity:create_grant(project_id=d78c4808482c4e74b6e8a3037b83de37, user_id=c04d978bfed6492fb377cd274964302b, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:11.714081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b0322-115b-4458-af78-8487d94ea670 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:11.717297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c67cc7-a9cd-456c-82a5-3deae4c4bbc9 None admin] RBAC: Authorizing `identity:get_user(user_id=cf6ad7faaf3247d5ba5962434f804797)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:11.718910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c67cc7-a9cd-456c-82a5-3deae4c4bbc9 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:11.720835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 66/134] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:11 2022] GET /identity/v3/users/cf6ad7faaf3247d5ba5962434f804797 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:11.727401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c67cc7-a9cd-456c-82a5-3deae4c4bbc9 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:11.752873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 69/135] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:08:11 2022] PUT /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37/users/c04d978bfed6492fb377cd274964302b/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:11.873564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b9774-62d1-4872-8d54-e7714c9b1ee4 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.873564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b9774-62d1-4872-8d54-e7714c9b1ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.873564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b9774-62d1-4872-8d54-e7714c9b1ee4 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.873846 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 70/136] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:08:11.875424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c67cc7-a9cd-456c-82a5-3deae4c4bbc9 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:11.881543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecfb8b-83a2-4527-959e-9f043e367969 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.881543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecfb8b-83a2-4527-959e-9f043e367969 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.881543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecfb8b-83a2-4527-959e-9f043e367969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.907501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f9ecfb8b-83a2-4527-959e-9f043e367969 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:11.908959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f9ecfb8b-83a2-4527-959e-9f043e367969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:11.970179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a85ab7-f120-44e4-9f93-e887b1a9f4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:11.970871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a85ab7-f120-44e4-9f93-e887b1a9f4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:11.971285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a85ab7-f120-44e4-9f93-e887b1a9f4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:11.972112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a85ab7-f120-44e4-9f93-e887b1a9f4f2 None admin] PATH_INFO: `/v3/projects/c94fd2abfa064be2b7a1d3a6af2bbb05` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:11.975438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a85ab7-f120-44e4-9f93-e887b1a9f4f2 None admin] RBAC: Authorizing `identity:get_project(project_id=c94fd2abfa064be2b7a1d3a6af2bbb05)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:11.975438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a85ab7-f120-44e4-9f93-e887b1a9f4f2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:11.977275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 67/137] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:11 2022] GET /identity/v3/projects/c94fd2abfa064be2b7a1d3a6af2bbb05 => generated 312 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:11.984725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a85ab7-f120-44e4-9f93-e887b1a9f4f2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:12.044085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 71/138] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:12.058616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:12.156461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a85ab7-f120-44e4-9f93-e887b1a9f4f2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:12.206466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679b847-cdaf-450c-be6d-d99c6755cafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:12.207218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679b847-cdaf-450c-be6d-d99c6755cafb None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:12.207420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679b847-cdaf-450c-be6d-d99c6755cafb None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:12.207694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679b847-cdaf-450c-be6d-d99c6755cafb None admin] PATH_INFO: `/v3/projects/c94fd2abfa064be2b7a1d3a6af2bbb05/users/cf6ad7faaf3247d5ba5962434f804797/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:12.209678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679b847-cdaf-450c-be6d-d99c6755cafb None admin] RBAC: Authorizing `identity:create_grant(project_id=c94fd2abfa064be2b7a1d3a6af2bbb05, user_id=cf6ad7faaf3247d5ba5962434f804797, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:12.211141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5679b847-cdaf-450c-be6d-d99c6755cafb None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:12.225378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 68/139] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:08:11 2022] PUT /identity/v3/projects/c94fd2abfa064be2b7a1d3a6af2bbb05/users/cf6ad7faaf3247d5ba5962434f804797/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:08:12.349185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecfb8b-83a2-4527-959e-9f043e367969 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:12.407344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:12.462648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecfb8b-83a2-4527-959e-9f043e367969 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:12.502159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378820b3-9f3a-4480-af9a-2eadb77e432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:12.503357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378820b3-9f3a-4480-af9a-2eadb77e432c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:12.503644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378820b3-9f3a-4480-af9a-2eadb77e432c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:12.503917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378820b3-9f3a-4480-af9a-2eadb77e432c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:12.505028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378820b3-9f3a-4480-af9a-2eadb77e432c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:12.506508 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378820b3-9f3a-4480-af9a-2eadb77e432c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:12.508250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 72/140] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:12.513659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679b847-cdaf-450c-be6d-d99c6755cafb None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:12.628277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:12.636631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679b847-cdaf-450c-be6d-d99c6755cafb None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:12.672874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a415d54-3713-4099-b1b7-a6489317ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:12.673401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a415d54-3713-4099-b1b7-a6489317ee34 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:12.673710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a415d54-3713-4099-b1b7-a6489317ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:12.673981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a415d54-3713-4099-b1b7-a6489317ee34 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:12.674661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a415d54-3713-4099-b1b7-a6489317ee34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:12.676644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a415d54-3713-4099-b1b7-a6489317ee34 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:12.707095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 69/141] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:08:12 2022] POST /identity/v3/users => generated 283 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:08:12.995498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5f817-a181-4913-8381-e4228298a78b None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:12.995730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5f817-a181-4913-8381-e4228298a78b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:12.996190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5f817-a181-4913-8381-e4228298a78b None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:12.997619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 73/142] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:13.009957 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08dec-425d-42af-8c00-f28a275874d7 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:13.010243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08dec-425d-42af-8c00-f28a275874d7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:13.010517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08dec-425d-42af-8c00-f28a275874d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:13.033886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-71a08dec-425d-42af-8c00-f28a275874d7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:13.035121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-71a08dec-425d-42af-8c00-f28a275874d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:13.125052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 70/143] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:13.524187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5f817-a181-4913-8381-e4228298a78b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:13.690257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5f817-a181-4913-8381-e4228298a78b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:13.717955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:13.750244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26d5a3-2ad2-4edd-91a8-70edcfc17305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:13.750895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26d5a3-2ad2-4edd-91a8-70edcfc17305 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:13.751175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26d5a3-2ad2-4edd-91a8-70edcfc17305 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:13.751448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26d5a3-2ad2-4edd-91a8-70edcfc17305 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:13.770913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26d5a3-2ad2-4edd-91a8-70edcfc17305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-fb26d5a3-2ad2-4edd-91a8-70edcfc17305 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:13.775857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:13.776679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:13.777441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 74/144] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:13.784940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08dec-425d-42af-8c00-f28a275874d7 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:13.965760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:13.998819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08dec-425d-42af-8c00-f28a275874d7 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:14.048892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1ced1-40e6-43a7-846b-081dd8a510fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:14.049436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1ced1-40e6-43a7-846b-081dd8a510fa None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:14.049817 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1ced1-40e6-43a7-846b-081dd8a510fa None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:14.050653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1ced1-40e6-43a7-846b-081dd8a510fa None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:14.050752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1ced1-40e6-43a7-846b-081dd8a510fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:14.051032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1ced1-40e6-43a7-846b-081dd8a510fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:14.052449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1ced1-40e6-43a7-846b-081dd8a510fa None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:14.061074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 71/145] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:14.069570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26d5a3-2ad2-4edd-91a8-70edcfc17305 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:14.198823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26d5a3-2ad2-4edd-91a8-70edcfc17305 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:14.238023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f0823-db7b-490e-a490-1cf9b752ce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:14.238665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f0823-db7b-490e-a490-1cf9b752ce5a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:14.238941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f0823-db7b-490e-a490-1cf9b752ce5a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:14.239213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f0823-db7b-490e-a490-1cf9b752ce5a None admin] PATH_INFO: `/v3/users/cf6ad7faaf3247d5ba5962434f804797` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:14.246228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f0823-db7b-490e-a490-1cf9b752ce5a None admin] RBAC: Authorizing `identity:get_user(user_id=cf6ad7faaf3247d5ba5962434f804797)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:14.247704 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f0823-db7b-490e-a490-1cf9b752ce5a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:14.249325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 75/146] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:14 2022] GET /identity/v3/users/cf6ad7faaf3247d5ba5962434f804797 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:14.255516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1ced1-40e6-43a7-846b-081dd8a510fa None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:14.382751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1ced1-40e6-43a7-846b-081dd8a510fa None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:14.429037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23b1a1-a4da-4f52-90c5-b4df56d7c5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:14.429699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23b1a1-a4da-4f52-90c5-b4df56d7c5ed None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:14.429983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23b1a1-a4da-4f52-90c5-b4df56d7c5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:14.430263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23b1a1-a4da-4f52-90c5-b4df56d7c5ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:14.442652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:14.514513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23b1a1-a4da-4f52-90c5-b4df56d7c5ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:14.516131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23b1a1-a4da-4f52-90c5-b4df56d7c5ed None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:14.635102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 72/147] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:14.640640 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f0823-db7b-490e-a490-1cf9b752ce5a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:14.687939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fd3ee-fff1-48d7-91cd-916e6b82b4fb None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:14.688143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fd3ee-fff1-48d7-91cd-916e6b82b4fb None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:14.688686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fd3ee-fff1-48d7-91cd-916e6b82b4fb None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:14.690229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 73/148] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:14.696164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4e71a-9018-446b-b08e-d751f430e519 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:14.696445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4e71a-9018-446b-b08e-d751f430e519 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:14.696742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4e71a-9018-446b-b08e-d751f430e519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:14.705890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:14.720682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b7a4e71a-9018-446b-b08e-d751f430e519 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:14.723979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b7a4e71a-9018-446b-b08e-d751f430e519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:14.779771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:14.781577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f0823-db7b-490e-a490-1cf9b752ce5a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:14.824493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20fb33-004e-4b2d-8a32-76b647266e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:14.825153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20fb33-004e-4b2d-8a32-76b647266e77 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:14.825412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20fb33-004e-4b2d-8a32-76b647266e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:14.825717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20fb33-004e-4b2d-8a32-76b647266e77 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:14.826854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20fb33-004e-4b2d-8a32-76b647266e77 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:14.828319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20fb33-004e-4b2d-8a32-76b647266e77 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:14.830175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 76/149] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:14 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:14.831490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 74/150] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:14.837959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20fb33-004e-4b2d-8a32-76b647266e77 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:14.957309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20fb33-004e-4b2d-8a32-76b647266e77 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:14.976507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:14.995404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f7d60-87c8-4ca1-9b4b-c34277fe94b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:14.996025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f7d60-87c8-4ca1-9b4b-c34277fe94b9 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:14.996306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f7d60-87c8-4ca1-9b4b-c34277fe94b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:14.996579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f7d60-87c8-4ca1-9b4b-c34277fe94b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:14.997316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f7d60-87c8-4ca1-9b4b-c34277fe94b9 None admin] RBAC: Adding query filter params (role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=d78c4808482c4e74b6e8a3037b83de37, user.id=cf6ad7faaf3247d5ba5962434f804797) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:14.997651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f7d60-87c8-4ca1-9b4b-c34277fe94b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:14.999090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f7d60-87c8-4ca1-9b4b-c34277fe94b9 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:15.005261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 77/151] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:14 2022] GET /identity/v3/role_assignments?user.id=cf6ad7faaf3247d5ba5962434f804797&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=84d6c85b41034cae925afeb8921381e8 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:15.492754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4e71a-9018-446b-b08e-d751f430e519 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:15.624027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea88e1-1586-44ac-a576-071dfb85cf85 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:15.624447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea88e1-1586-44ac-a576-071dfb85cf85 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:15.624555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea88e1-1586-44ac-a576-071dfb85cf85 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:15.626059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 78/152] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:15.631030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4e71a-9018-446b-b08e-d751f430e519 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:15.638887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564322d-f5ff-4564-9b1e-ed11b857bbf5 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:15.639168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564322d-f5ff-4564-9b1e-ed11b857bbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:15.639447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564322d-f5ff-4564-9b1e-ed11b857bbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:15.678606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9564322d-f5ff-4564-9b1e-ed11b857bbf5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:15.679828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9564322d-f5ff-4564-9b1e-ed11b857bbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:15.683264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce434051-4a69-4ef3-a86b-5d96db60341c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:15.684198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce434051-4a69-4ef3-a86b-5d96db60341c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:15.684689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce434051-4a69-4ef3-a86b-5d96db60341c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:15.685163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce434051-4a69-4ef3-a86b-5d96db60341c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:15.701798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce434051-4a69-4ef3-a86b-5d96db60341c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-ce434051-4a69-4ef3-a86b-5d96db60341c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:15.707508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:15.708403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:15.709169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:15.712012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 75/153] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:15.719748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce434051-4a69-4ef3-a86b-5d96db60341c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:15.759865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:15.832606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 79/154] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:15.845053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce434051-4a69-4ef3-a86b-5d96db60341c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:15.888578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd6143-cb5f-4275-9939-dc0ad4f71bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:15.889251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd6143-cb5f-4275-9939-dc0ad4f71bcd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:15.889556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd6143-cb5f-4275-9939-dc0ad4f71bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:15.889882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd6143-cb5f-4275-9939-dc0ad4f71bcd None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:15.890563 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd6143-cb5f-4275-9939-dc0ad4f71bcd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:15.890868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd6143-cb5f-4275-9939-dc0ad4f71bcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:15.892333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd6143-cb5f-4275-9939-dc0ad4f71bcd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:15.902201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 76/155] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:15.907779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564322d-f5ff-4564-9b1e-ed11b857bbf5 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:15.959965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d99ee-c0dc-4b5b-8b8f-6a4d34313727 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:15.960706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d99ee-c0dc-4b5b-8b8f-6a4d34313727 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:15.961414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d99ee-c0dc-4b5b-8b8f-6a4d34313727 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:15.963588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 77/156] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:15.964708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbcfcf-ea5e-4625-897e-522906df546a None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:15.964994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbcfcf-ea5e-4625-897e-522906df546a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:15.965268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbcfcf-ea5e-4625-897e-522906df546a None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:15.967367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 78/157] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:15.973468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e53f2d-d31b-4121-9105-1e479e2ccfc7 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:15.973775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e53f2d-d31b-4121-9105-1e479e2ccfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:15.974051 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e53f2d-d31b-4121-9105-1e479e2ccfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:16.001736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-34e53f2d-d31b-4121-9105-1e479e2ccfc7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:16.002982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-34e53f2d-d31b-4121-9105-1e479e2ccfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:16.076850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564322d-f5ff-4564-9b1e-ed11b857bbf5 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:16.094948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:16.140203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 79/158] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:16.141552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358068d-69a1-4fd1-9de3-28c0a883742b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:16.142811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358068d-69a1-4fd1-9de3-28c0a883742b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:16.143087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358068d-69a1-4fd1-9de3-28c0a883742b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:16.170292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7c4c-ec30-4f41-8b57-018ca707300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:16.170940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7c4c-ec30-4f41-8b57-018ca707300d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:16.171217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7c4c-ec30-4f41-8b57-018ca707300d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:16.171486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7c4c-ec30-4f41-8b57-018ca707300d None admin] PATH_INFO: `/v3/users/cf6ad7faaf3247d5ba5962434f804797` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:16.178183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a7c4c-ec30-4f41-8b57-018ca707300d None admin] RBAC: Authorizing `identity:get_user(user_id=cf6ad7faaf3247d5ba5962434f804797)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:16.179631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a7c4c-ec30-4f41-8b57-018ca707300d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:16.181216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 80/159] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:15 2022] GET /identity/v3/users/cf6ad7faaf3247d5ba5962434f804797 => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:16.182434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1358068d-69a1-4fd1-9de3-28c0a883742b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:16.183625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1358068d-69a1-4fd1-9de3-28c0a883742b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:16.186532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7c4c-ec30-4f41-8b57-018ca707300d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:16.219422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:16.306832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 80/160] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:16.335211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358068d-69a1-4fd1-9de3-28c0a883742b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:16.375811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7c4c-ec30-4f41-8b57-018ca707300d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:16.413821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2695e12-dc09-4fed-96e1-ce854310ec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:16.414436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2695e12-dc09-4fed-96e1-ce854310ec57 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:16.414722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2695e12-dc09-4fed-96e1-ce854310ec57 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:16.415003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2695e12-dc09-4fed-96e1-ce854310ec57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:16.474056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:16.491350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2695e12-dc09-4fed-96e1-ce854310ec57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:16.493039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2695e12-dc09-4fed-96e1-ce854310ec57 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:16.543149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358068d-69a1-4fd1-9de3-28c0a883742b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:16.601701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 81/161] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 20:08:16.606458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49515951-207b-43ad-bdf7-cebb4f7166ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:16.606811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2695e12-dc09-4fed-96e1-ce854310ec57 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:16.607100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49515951-207b-43ad-bdf7-cebb4f7166ff None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:16.607376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49515951-207b-43ad-bdf7-cebb4f7166ff None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:16.607643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49515951-207b-43ad-bdf7-cebb4f7166ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:16.622498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49515951-207b-43ad-bdf7-cebb4f7166ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-49515951-207b-43ad-bdf7-cebb4f7166ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:16.627267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:16.628747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:16.629489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:16.633977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 81/162] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:16.642433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49515951-207b-43ad-bdf7-cebb4f7166ff None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:16.661323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:16.697896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:16.725966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2695e12-dc09-4fed-96e1-ce854310ec57 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:16.771729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce657-dc6d-4bf5-854d-ed6eaf3e67b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:16.772758 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce657-dc6d-4bf5-854d-ed6eaf3e67b2 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:16.773234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce657-dc6d-4bf5-854d-ed6eaf3e67b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:16.777965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ce657-dc6d-4bf5-854d-ed6eaf3e67b2 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:16.779571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ce657-dc6d-4bf5-854d-ed6eaf3e67b2 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:16.781389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ce657-dc6d-4bf5-854d-ed6eaf3e67b2 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:16.783413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 82/163] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:16 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:16.789874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce657-dc6d-4bf5-854d-ed6eaf3e67b2 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:16.805553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49515951-207b-43ad-bdf7-cebb4f7166ff None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:16.864935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab44d4d-e4c6-4599-9938-b70e1dde4bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:16.865578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab44d4d-e4c6-4599-9938-b70e1dde4bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:16.865894 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab44d4d-e4c6-4599-9938-b70e1dde4bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:16.865999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab44d4d-e4c6-4599-9938-b70e1dde4bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:16.866676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab44d4d-e4c6-4599-9938-b70e1dde4bb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:16.866977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab44d4d-e4c6-4599-9938-b70e1dde4bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:16.868382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab44d4d-e4c6-4599-9938-b70e1dde4bb1 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:16.885845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 82/164] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:16.886442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab44d4d-e4c6-4599-9938-b70e1dde4bb1 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:16.928033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ce657-dc6d-4bf5-854d-ed6eaf3e67b2 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:16.999695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46655b8b-489c-4ff3-87fd-a5450d42d01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.000927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46655b8b-489c-4ff3-87fd-a5450d42d01a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.001962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46655b8b-489c-4ff3-87fd-a5450d42d01a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.003367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46655b8b-489c-4ff3-87fd-a5450d42d01a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.022802 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46655b8b-489c-4ff3-87fd-a5450d42d01a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-46655b8b-489c-4ff3-87fd-a5450d42d01a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:17.031276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:17.032108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:17.032859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:17.041085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 83/165] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:17.041863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:17.044323 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46655b8b-489c-4ff3-87fd-a5450d42d01a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:17.061900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:17.103656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab44d4d-e4c6-4599-9938-b70e1dde4bb1 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.161485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46655b8b-489c-4ff3-87fd-a5450d42d01a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.178524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd86b7-dd22-4c0d-8989-ac572e55ec3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.179453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd86b7-dd22-4c0d-8989-ac572e55ec3a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.179885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd86b7-dd22-4c0d-8989-ac572e55ec3a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.180396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd86b7-dd22-4c0d-8989-ac572e55ec3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.181254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd86b7-dd22-4c0d-8989-ac572e55ec3a None admin] RBAC: Adding query filter params (role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=d78c4808482c4e74b6e8a3037b83de37, user.id=cf6ad7faaf3247d5ba5962434f804797) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:17.181730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd86b7-dd22-4c0d-8989-ac572e55ec3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:17.183423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd86b7-dd22-4c0d-8989-ac572e55ec3a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:17.198055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390084b-275e-453f-bcf4-df5d96add05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.198747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390084b-275e-453f-bcf4-df5d96add05b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.199032 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390084b-275e-453f-bcf4-df5d96add05b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.199308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390084b-275e-453f-bcf4-df5d96add05b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.202167 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 83/166] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:16 2022] GET /identity/v3/role_assignments?user.id=cf6ad7faaf3247d5ba5962434f804797&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:17.203055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd86b7-dd22-4c0d-8989-ac572e55ec3a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:17.211114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390084b-275e-453f-bcf4-df5d96add05b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-2390084b-275e-453f-bcf4-df5d96add05b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:17.216344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:17.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:17.218052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 84/167] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:08:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:17.218767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390084b-275e-453f-bcf4-df5d96add05b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:17.297960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:17.308851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd86b7-dd22-4c0d-8989-ac572e55ec3a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.346003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390084b-275e-453f-bcf4-df5d96add05b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.354467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d31db-2dde-4225-a0ef-cb9529ef53d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.355122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d31db-2dde-4225-a0ef-cb9529ef53d2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.355399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d31db-2dde-4225-a0ef-cb9529ef53d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.355669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d31db-2dde-4225-a0ef-cb9529ef53d2 None admin] PATH_INFO: `/v3/users/c5ff4a05676249e282fa93d97fcd6eb9` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.356852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d31db-2dde-4225-a0ef-cb9529ef53d2 None admin] RBAC: Authorizing `identity:get_user(user_id=c5ff4a05676249e282fa93d97fcd6eb9)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:17.358382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d31db-2dde-4225-a0ef-cb9529ef53d2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:17.360196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 84/168] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:17 2022] GET /identity/v3/users/c5ff4a05676249e282fa93d97fcd6eb9 => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:17.361341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3bfbd-3589-41b2-9fdd-ed7cdb9cc6fb None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.361656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3bfbd-3589-41b2-9fdd-ed7cdb9cc6fb None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.361925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3bfbd-3589-41b2-9fdd-ed7cdb9cc6fb None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.363926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 85/169] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:17.364549 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3bfbd-3589-41b2-9fdd-ed7cdb9cc6fb None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:17.385899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:17.413736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcfec2-0ed0-4718-a352-97aac996e78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.414070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcfec2-0ed0-4718-a352-97aac996e78f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.414354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcfec2-0ed0-4718-a352-97aac996e78f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.414715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcfec2-0ed0-4718-a352-97aac996e78f None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.415439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcfec2-0ed0-4718-a352-97aac996e78f None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:17.415797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcfec2-0ed0-4718-a352-97aac996e78f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:17.417252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcfec2-0ed0-4718-a352-97aac996e78f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:17.432972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 85/170] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:17.434323 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5918f5-e633-4ad4-a50f-2ce5792e3b23 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.434607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5918f5-e633-4ad4-a50f-2ce5792e3b23 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.434882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5918f5-e633-4ad4-a50f-2ce5792e3b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.459303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8e5918f5-e633-4ad4-a50f-2ce5792e3b23 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:17.460657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8e5918f5-e633-4ad4-a50f-2ce5792e3b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:17.479184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3bfbd-3589-41b2-9fdd-ed7cdb9cc6fb None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.517852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b498783-230f-4bf0-abee-36ac4ca19b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.518635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b498783-230f-4bf0-abee-36ac4ca19b47 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.518915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b498783-230f-4bf0-abee-36ac4ca19b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.519206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b498783-230f-4bf0-abee-36ac4ca19b47 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.519973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b498783-230f-4bf0-abee-36ac4ca19b47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:17.520314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b498783-230f-4bf0-abee-36ac4ca19b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:17.521740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b498783-230f-4bf0-abee-36ac4ca19b47 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:17.530657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 86/171] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:17.531559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b498783-230f-4bf0-abee-36ac4ca19b47 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:17.538305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 86/172] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:17.539250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5918f5-e633-4ad4-a50f-2ce5792e3b23 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:17.608018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:17.610731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:17.642002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b498783-230f-4bf0-abee-36ac4ca19b47 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.676334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5918f5-e633-4ad4-a50f-2ce5792e3b23 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.678024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2d3a5-0bc1-4c4d-ad62-963cbb4760de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.678661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2d3a5-0bc1-4c4d-ad62-963cbb4760de None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.678940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2d3a5-0bc1-4c4d-ad62-963cbb4760de None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.679211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2d3a5-0bc1-4c4d-ad62-963cbb4760de None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.680329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2d3a5-0bc1-4c4d-ad62-963cbb4760de None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:17.681912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2d3a5-0bc1-4c4d-ad62-963cbb4760de None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:17.683562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 87/173] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:17 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:17.684411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2d3a5-0bc1-4c4d-ad62-963cbb4760de None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:17.726298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37271711-a4a2-48b0-8454-5446375a6d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.726950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37271711-a4a2-48b0-8454-5446375a6d4f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.727231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37271711-a4a2-48b0-8454-5446375a6d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.727507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37271711-a4a2-48b0-8454-5446375a6d4f None admin] PATH_INFO: `/v3/users/cf6ad7faaf3247d5ba5962434f804797` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.729429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37271711-a4a2-48b0-8454-5446375a6d4f None admin] RBAC: Authorizing `identity:get_user(user_id=cf6ad7faaf3247d5ba5962434f804797)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:17.733083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37271711-a4a2-48b0-8454-5446375a6d4f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:17.734856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 87/174] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:17 2022] GET /identity/v3/users/cf6ad7faaf3247d5ba5962434f804797 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:17.735622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37271711-a4a2-48b0-8454-5446375a6d4f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:17.792250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2d3a5-0bc1-4c4d-ad62-963cbb4760de None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.827135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553725a-f8a2-420e-acdb-2b2649a5d06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.827769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553725a-f8a2-420e-acdb-2b2649a5d06d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.828043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553725a-f8a2-420e-acdb-2b2649a5d06d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.828307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553725a-f8a2-420e-acdb-2b2649a5d06d None admin] PATH_INFO: `/v3/users/c04d978bfed6492fb377cd274964302b` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.829430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553725a-f8a2-420e-acdb-2b2649a5d06d None admin] RBAC: Authorizing `identity:get_user(user_id=c04d978bfed6492fb377cd274964302b)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:17.831009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553725a-f8a2-420e-acdb-2b2649a5d06d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:17.832571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 88/175] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:17 2022] GET /identity/v3/users/c04d978bfed6492fb377cd274964302b => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:17.833404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553725a-f8a2-420e-acdb-2b2649a5d06d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:17.837966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:17.896023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37271711-a4a2-48b0-8454-5446375a6d4f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.942632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553725a-f8a2-420e-acdb-2b2649a5d06d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:17.949867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:17.965274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683755c-14b3-4870-9311-f3e45ccf041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.965975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683755c-14b3-4870-9311-f3e45ccf041b None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.966257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683755c-14b3-4870-9311-f3e45ccf041b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.966535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683755c-14b3-4870-9311-f3e45ccf041b None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37/users/c5ff4a05676249e282fa93d97fcd6eb9/roles/853e41e951d14277bb9e21a8861bdbf9` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.971028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683755c-14b3-4870-9311-f3e45ccf041b None admin] RBAC: Authorizing `identity:create_grant(project_id=d78c4808482c4e74b6e8a3037b83de37, user_id=c5ff4a05676249e282fa93d97fcd6eb9, role_id=853e41e951d14277bb9e21a8861bdbf9)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:17.973018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683755c-14b3-4870-9311-f3e45ccf041b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:17.978701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ace30-0a9a-4254-aabb-5e4ff657c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:17.979373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ace30-0a9a-4254-aabb-5e4ff657c95e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:17.979709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ace30-0a9a-4254-aabb-5e4ff657c95e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:17.979977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ace30-0a9a-4254-aabb-5e4ff657c95e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:17.999544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 88/176] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:08:17 2022] PUT /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37/users/c5ff4a05676249e282fa93d97fcd6eb9/roles/853e41e951d14277bb9e21a8861bdbf9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:18.000434 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683755c-14b3-4870-9311-f3e45ccf041b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:18.066474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ace30-0a9a-4254-aabb-5e4ff657c95e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:18.068028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ace30-0a9a-4254-aabb-5e4ff657c95e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:18.071261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:18.185813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 89/177] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:18.187093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84261-98ac-44e3-8719-c5e275a8ee25 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.187379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84261-98ac-44e3-8719-c5e275a8ee25 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.187667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a84261-98ac-44e3-8719-c5e275a8ee25 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.189277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 90/178] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:18.189874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84261-98ac-44e3-8719-c5e275a8ee25 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:18.218169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683755c-14b3-4870-9311-f3e45ccf041b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:18.261303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997cbbf-b32a-4a4b-82e6-e43e07b30dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:18.262140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997cbbf-b32a-4a4b-82e6-e43e07b30dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.262365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997cbbf-b32a-4a4b-82e6-e43e07b30dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.262637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997cbbf-b32a-4a4b-82e6-e43e07b30dc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.301482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a84261-98ac-44e3-8719-c5e275a8ee25 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:18.304635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:18.324400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:18.338111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93019df-ebf6-45b9-a46f-a2c6640b7ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:18.338744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93019df-ebf6-45b9-a46f-a2c6640b7ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.339022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93019df-ebf6-45b9-a46f-a2c6640b7ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.339282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93019df-ebf6-45b9-a46f-a2c6640b7ec8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.340437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997cbbf-b32a-4a4b-82e6-e43e07b30dc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:18.342037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997cbbf-b32a-4a4b-82e6-e43e07b30dc1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:18.349996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93019df-ebf6-45b9-a46f-a2c6640b7ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-a93019df-ebf6-45b9-a46f-a2c6640b7ec8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:18.354801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:18.355653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:18.356423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 91/179] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:08:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:18.357701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95f892-2011-488b-a735-a04a7adc47d3 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.357800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95f892-2011-488b-a735-a04a7adc47d3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.358059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95f892-2011-488b-a735-a04a7adc47d3 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.359410 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 92/180] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:18.360283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3710629-df3b-4cdf-aae2-ea573042fbe5 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.360558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3710629-df3b-4cdf-aae2-ea573042fbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.360825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3710629-df3b-4cdf-aae2-ea573042fbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.381210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a3710629-df3b-4cdf-aae2-ea573042fbe5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:18.383001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a3710629-df3b-4cdf-aae2-ea573042fbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:18.456150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 89/181] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:18.456678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 93/182] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:18.457303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997cbbf-b32a-4a4b-82e6-e43e07b30dc1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:18.457933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3710629-df3b-4cdf-aae2-ea573042fbe5 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:18.566055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:18.574679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:18.574964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3710629-df3b-4cdf-aae2-ea573042fbe5 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:18.593811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997cbbf-b32a-4a4b-82e6-e43e07b30dc1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:18.619923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df8e9f-4fbf-409c-9a1e-e8732781d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:18.620546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df8e9f-4fbf-409c-9a1e-e8732781d05f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.621137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df8e9f-4fbf-409c-9a1e-e8732781d05f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.621418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df8e9f-4fbf-409c-9a1e-e8732781d05f None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.622390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df8e9f-4fbf-409c-9a1e-e8732781d05f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:18.622695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df8e9f-4fbf-409c-9a1e-e8732781d05f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:18.624274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df8e9f-4fbf-409c-9a1e-e8732781d05f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:18.632960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb552fa5-d7d5-4676-9e2d-489a92429327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:18.633893 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb552fa5-d7d5-4676-9e2d-489a92429327 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.634361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb552fa5-d7d5-4676-9e2d-489a92429327 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.634818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb552fa5-d7d5-4676-9e2d-489a92429327 None admin] PATH_INFO: `/v3/projects/c94fd2abfa064be2b7a1d3a6af2bbb05` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.637653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 94/183] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:18.639225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd85b6-6b98-4da6-824f-7e99bb238ca6 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.639503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd85b6-6b98-4da6-824f-7e99bb238ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.639770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd85b6-6b98-4da6-824f-7e99bb238ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.642332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb552fa5-d7d5-4676-9e2d-489a92429327 None admin] RBAC: Authorizing `identity:get_project(project_id=c94fd2abfa064be2b7a1d3a6af2bbb05)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:18.644055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb552fa5-d7d5-4676-9e2d-489a92429327 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:18.646184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 90/184] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:18 2022] GET /identity/v3/projects/c94fd2abfa064be2b7a1d3a6af2bbb05 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:18.648021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb552fa5-d7d5-4676-9e2d-489a92429327 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:18.660135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-19bd85b6-6b98-4da6-824f-7e99bb238ca6 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:18.661321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-19bd85b6-6b98-4da6-824f-7e99bb238ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:18.740813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 95/185] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:18.741487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd85b6-6b98-4da6-824f-7e99bb238ca6 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:18.767281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb552fa5-d7d5-4676-9e2d-489a92429327 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:18.806600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4578dd-bc21-485f-a8c8-4b63231d1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:18.807684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4578dd-bc21-485f-a8c8-4b63231d1e0a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.808202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4578dd-bc21-485f-a8c8-4b63231d1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.808729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4578dd-bc21-485f-a8c8-4b63231d1e0a None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.810083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4578dd-bc21-485f-a8c8-4b63231d1e0a None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:18.812879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4578dd-bc21-485f-a8c8-4b63231d1e0a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:18.815037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 91/186] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:18 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:18.816113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4578dd-bc21-485f-a8c8-4b63231d1e0a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:18.833804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:18.847571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:18.850866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd85b6-6b98-4da6-824f-7e99bb238ca6 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:18.888242 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2663cb-f9a7-49fc-8e62-073d68cd4f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:18.889019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2663cb-f9a7-49fc-8e62-073d68cd4f10 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.889245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2663cb-f9a7-49fc-8e62-073d68cd4f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.889535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2663cb-f9a7-49fc-8e62-073d68cd4f10 None admin] PATH_INFO: `/v3/users/0e83f81e6d634210aab861ccc61a0ab4` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.904658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2663cb-f9a7-49fc-8e62-073d68cd4f10 None admin] RBAC: Authorizing `identity:get_user(user_id=0e83f81e6d634210aab861ccc61a0ab4)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:18.906220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2663cb-f9a7-49fc-8e62-073d68cd4f10 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:18.913065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 96/187] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:18 2022] GET /identity/v3/users/0e83f81e6d634210aab861ccc61a0ab4 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:18.913065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2663cb-f9a7-49fc-8e62-073d68cd4f10 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:18.946236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4578dd-bc21-485f-a8c8-4b63231d1e0a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:18.987563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd02e65-9a25-4fe1-81b2-fa508deb544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:18.988842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd02e65-9a25-4fe1-81b2-fa508deb544f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:18.989351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd02e65-9a25-4fe1-81b2-fa508deb544f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:18.989857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd02e65-9a25-4fe1-81b2-fa508deb544f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:18.990780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd02e65-9a25-4fe1-81b2-fa508deb544f None admin] RBAC: Adding query filter params (role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=c94fd2abfa064be2b7a1d3a6af2bbb05, user.id=cf6ad7faaf3247d5ba5962434f804797) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:18.991287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd02e65-9a25-4fe1-81b2-fa508deb544f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:18.992936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd02e65-9a25-4fe1-81b2-fa508deb544f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:18.999125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 92/188] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:18 2022] GET /identity/v3/role_assignments?user.id=cf6ad7faaf3247d5ba5962434f804797&scope.project.id=c94fd2abfa064be2b7a1d3a6af2bbb05&role.id=84d6c85b41034cae925afeb8921381e8 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:19.000133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd02e65-9a25-4fe1-81b2-fa508deb544f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:19.024206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2663cb-f9a7-49fc-8e62-073d68cd4f10 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.063875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a3f93-ee92-42cc-80bd-052d161936b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:19.064524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a3f93-ee92-42cc-80bd-052d161936b5 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:19.064909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a3f93-ee92-42cc-80bd-052d161936b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:19.065830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a3f93-ee92-42cc-80bd-052d161936b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:19.067146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a3f93-ee92-42cc-80bd-052d161936b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:19.068559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a3f93-ee92-42cc-80bd-052d161936b5 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:19.070592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 97/189] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:19.071216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a3f93-ee92-42cc-80bd-052d161936b5 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:19.088417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:19.105848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:19.128440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd02e65-9a25-4fe1-81b2-fa508deb544f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.174408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0158a-198e-436e-8bab-552bf7d66caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:19.176076 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0158a-198e-436e-8bab-552bf7d66caf None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:19.176555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0158a-198e-436e-8bab-552bf7d66caf None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:19.177084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0158a-198e-436e-8bab-552bf7d66caf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:19.178064 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0158a-198e-436e-8bab-552bf7d66caf None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=d78c4808482c4e74b6e8a3037b83de37, user.id=c04d978bfed6492fb377cd274964302b) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:19.178576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0158a-198e-436e-8bab-552bf7d66caf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:19.181163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0158a-198e-436e-8bab-552bf7d66caf None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:19.189081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 93/190] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:18 2022] GET /identity/v3/role_assignments?user.id=c04d978bfed6492fb377cd274964302b&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=36331ec562ff42f883fc883363bc4c8d => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:19.190331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0158a-198e-436e-8bab-552bf7d66caf None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:19.220078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a3f93-ee92-42cc-80bd-052d161936b5 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.260281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720cbc9-eb68-4fcf-9ed5-d1eeb9662edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:19.261288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720cbc9-eb68-4fcf-9ed5-d1eeb9662edd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:19.261773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720cbc9-eb68-4fcf-9ed5-d1eeb9662edd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:19.262225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720cbc9-eb68-4fcf-9ed5-d1eeb9662edd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:19.341451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720cbc9-eb68-4fcf-9ed5-d1eeb9662edd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:19.343284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720cbc9-eb68-4fcf-9ed5-d1eeb9662edd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:19.346119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0158a-198e-436e-8bab-552bf7d66caf None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.361076 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:19.386153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf2354-cbc9-42d3-b737-d8d104e33566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:19.386809 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf2354-cbc9-42d3-b737-d8d104e33566 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:19.387088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf2354-cbc9-42d3-b737-d8d104e33566 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:19.387385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf2354-cbc9-42d3-b737-d8d104e33566 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:19.401034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf2354-cbc9-42d3-b737-d8d104e33566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-23cf2354-cbc9-42d3-b737-d8d104e33566 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:19.407265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:19.408183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:19.408980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 94/191] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:19.409920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf2354-cbc9-42d3-b737-d8d104e33566 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:19.423497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:19.475877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 98/192] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:19.477018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720cbc9-eb68-4fcf-9ed5-d1eeb9662edd None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:19.547321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf2354-cbc9-42d3-b737-d8d104e33566 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.588595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87f38-c516-4e50-aaac-bca596089838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:19.589209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87f38-c516-4e50-aaac-bca596089838 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:19.589486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87f38-c516-4e50-aaac-bca596089838 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:19.589790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff87f38-c516-4e50-aaac-bca596089838 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:19.590352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720cbc9-eb68-4fcf-9ed5-d1eeb9662edd None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.590465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff87f38-c516-4e50-aaac-bca596089838 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:19.591998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff87f38-c516-4e50-aaac-bca596089838 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:19.603345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:19.616758 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 95/193] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:08:19 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:08:19.618022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87f38-c516-4e50-aaac-bca596089838 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:19.636162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dde46-de56-4e03-b446-9bb4ea116bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:19.637145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dde46-de56-4e03-b446-9bb4ea116bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:19.637641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dde46-de56-4e03-b446-9bb4ea116bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:19.638066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dde46-de56-4e03-b446-9bb4ea116bd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:19.638944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96dde46-de56-4e03-b446-9bb4ea116bd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:19.639396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96dde46-de56-4e03-b446-9bb4ea116bd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:19.641347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96dde46-de56-4e03-b446-9bb4ea116bd4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:19.652649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 99/194] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:19.658302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dde46-de56-4e03-b446-9bb4ea116bd4 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:19.716246 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:19.762316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff87f38-c516-4e50-aaac-bca596089838 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.779509 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dde46-de56-4e03-b446-9bb4ea116bd4 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.807604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af572f-0a14-4add-99bc-3ea5be75bf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:19.809642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af572f-0a14-4add-99bc-3ea5be75bf2e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:19.810358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af572f-0a14-4add-99bc-3ea5be75bf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:19.810784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af572f-0a14-4add-99bc-3ea5be75bf2e None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:19.812920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af572f-0a14-4add-99bc-3ea5be75bf2e None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:19.815971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af572f-0a14-4add-99bc-3ea5be75bf2e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:19.822435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c5b5d-de30-4c8f-b811-a5ca8cb4d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:19.822435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c5b5d-de30-4c8f-b811-a5ca8cb4d773 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:19.822435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c5b5d-de30-4c8f-b811-a5ca8cb4d773 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:19.822435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c5b5d-de30-4c8f-b811-a5ca8cb4d773 None admin] PATH_INFO: `/v3/users/0e83f81e6d634210aab861ccc61a0ab4` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:19.822827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 96/195] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:19 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:19.824390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c5b5d-de30-4c8f-b811-a5ca8cb4d773 None admin] RBAC: Authorizing `identity:get_user(user_id=0e83f81e6d634210aab861ccc61a0ab4)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:19.825102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c5b5d-de30-4c8f-b811-a5ca8cb4d773 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:19.826523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af572f-0a14-4add-99bc-3ea5be75bf2e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:19.826976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 100/196] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:19 2022] GET /identity/v3/users/0e83f81e6d634210aab861ccc61a0ab4 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:19.834068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c5b5d-de30-4c8f-b811-a5ca8cb4d773 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:19.886340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:19.951827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c5b5d-de30-4c8f-b811-a5ca8cb4d773 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.966046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af572f-0a14-4add-99bc-3ea5be75bf2e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:19.995355 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc17d8-f1dd-4bc7-a40a-b3a8647b0b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:19.995973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc17d8-f1dd-4bc7-a40a-b3a8647b0b36 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:19.996245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc17d8-f1dd-4bc7-a40a-b3a8647b0b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:19.996511 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc17d8-f1dd-4bc7-a40a-b3a8647b0b36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:20.010118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44029b0-e66e-4434-a6aa-923756f0ca10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:20.010733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44029b0-e66e-4434-a6aa-923756f0ca10 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:20.011011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44029b0-e66e-4434-a6aa-923756f0ca10 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:20.011292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44029b0-e66e-4434-a6aa-923756f0ca10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:20.012023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44029b0-e66e-4434-a6aa-923756f0ca10 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919, user.id=0e83f81e6d634210aab861ccc61a0ab4) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:20.012335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44029b0-e66e-4434-a6aa-923756f0ca10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:20.014116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44029b0-e66e-4434-a6aa-923756f0ca10 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:20.016906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:20.029214 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 97/197] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:19 2022] GET /identity/v3/role_assignments?user.id=0e83f81e6d634210aab861ccc61a0ab4&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=36331ec562ff42f883fc883363bc4c8d => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:20.087292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc17d8-f1dd-4bc7-a40a-b3a8647b0b36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:20.088795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc17d8-f1dd-4bc7-a40a-b3a8647b0b36 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:20.142311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:20.231538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 101/198] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:20.237771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44029b0-e66e-4434-a6aa-923756f0ca10 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:20.320354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c518b8-65c3-483b-a0c3-e3a87ad94b71 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:20.320670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c518b8-65c3-483b-a0c3-e3a87ad94b71 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:20.320899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c518b8-65c3-483b-a0c3-e3a87ad94b71 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:20.322507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 102/199] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:20.327606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30998db2-036b-407d-b6a6-9f6386182b10 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:20.327882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30998db2-036b-407d-b6a6-9f6386182b10 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:20.328151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30998db2-036b-407d-b6a6-9f6386182b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:20.352044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-30998db2-036b-407d-b6a6-9f6386182b10 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:20.353236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-30998db2-036b-407d-b6a6-9f6386182b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:20.456127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44029b0-e66e-4434-a6aa-923756f0ca10 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:20.478415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 103/200] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:20.511058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03e313-95cf-4a62-872b-afab353f743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:20.512082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e313-95cf-4a62-872b-afab353f743c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:20.512551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e313-95cf-4a62-872b-afab353f743c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:20.513012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e313-95cf-4a62-872b-afab353f743c None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:20.514531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03e313-95cf-4a62-872b-afab353f743c None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:20.516408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03e313-95cf-4a62-872b-afab353f743c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:20.520517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 98/201] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:20 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:20.524087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30998db2-036b-407d-b6a6-9f6386182b10 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:20.617253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:20.649181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30998db2-036b-407d-b6a6-9f6386182b10 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:20.688763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad16ec-29bc-491a-b502-1ef158ef9dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:20.689366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad16ec-29bc-491a-b502-1ef158ef9dec None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:20.689682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad16ec-29bc-491a-b502-1ef158ef9dec None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:20.689950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad16ec-29bc-491a-b502-1ef158ef9dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:20.690759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad16ec-29bc-491a-b502-1ef158ef9dec None admin] RBAC: Adding query filter params (role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919, user.id=0e83f81e6d634210aab861ccc61a0ab4) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:20.691062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad16ec-29bc-491a-b502-1ef158ef9dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:20.692559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad16ec-29bc-491a-b502-1ef158ef9dec None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:20.700731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 104/202] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:20 2022] GET /identity/v3/role_assignments?user.id=0e83f81e6d634210aab861ccc61a0ab4&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:20.756455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03e313-95cf-4a62-872b-afab353f743c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:20.842814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:20.897090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03e313-95cf-4a62-872b-afab353f743c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:20.945466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92204ae-d7d5-41f5-bbd6-24803e6f6baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:20.946127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92204ae-d7d5-41f5-bbd6-24803e6f6baf None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:20.946400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92204ae-d7d5-41f5-bbd6-24803e6f6baf None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:20.946667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92204ae-d7d5-41f5-bbd6-24803e6f6baf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:20.959148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92204ae-d7d5-41f5-bbd6-24803e6f6baf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-a92204ae-d7d5-41f5-bbd6-24803e6f6baf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:20.964444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:20.965256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:20.966463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 99/203] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:20.972174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad16ec-29bc-491a-b502-1ef158ef9dec None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:21.092976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad16ec-29bc-491a-b502-1ef158ef9dec None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:21.138075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e578f-bb62-4e77-9d8a-b8afdd1acac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:21.138965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e578f-bb62-4e77-9d8a-b8afdd1acac1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:21.139444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e578f-bb62-4e77-9d8a-b8afdd1acac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:21.139884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e578f-bb62-4e77-9d8a-b8afdd1acac1 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:21.140707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e578f-bb62-4e77-9d8a-b8afdd1acac1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:21.141197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e578f-bb62-4e77-9d8a-b8afdd1acac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:21.142757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e578f-bb62-4e77-9d8a-b8afdd1acac1 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:21.146544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:21.159531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 105/204] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:21.165208 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92204ae-d7d5-41f5-bbd6-24803e6f6baf None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:21.283324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92204ae-d7d5-41f5-bbd6-24803e6f6baf None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:21.309840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:21.319581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de81004-6aef-48d0-902d-6d2eb34ed207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:21.320223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de81004-6aef-48d0-902d-6d2eb34ed207 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:21.320875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de81004-6aef-48d0-902d-6d2eb34ed207 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:21.321531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de81004-6aef-48d0-902d-6d2eb34ed207 None admin] PATH_INFO: `/v3/users/c5ff4a05676249e282fa93d97fcd6eb9` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:21.323011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de81004-6aef-48d0-902d-6d2eb34ed207 None admin] RBAC: Authorizing `identity:get_user(user_id=c5ff4a05676249e282fa93d97fcd6eb9)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:21.324453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de81004-6aef-48d0-902d-6d2eb34ed207 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:21.326372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 100/205] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:21 2022] GET /identity/v3/users/c5ff4a05676249e282fa93d97fcd6eb9 => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:21.331661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e578f-bb62-4e77-9d8a-b8afdd1acac1 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:21.468608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e578f-bb62-4e77-9d8a-b8afdd1acac1 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:21.503609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff0e47-92f6-43ff-9de1-f63d6cc06414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:21.504217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0e47-92f6-43ff-9de1-f63d6cc06414 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:21.504541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0e47-92f6-43ff-9de1-f63d6cc06414 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:21.504813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff0e47-92f6-43ff-9de1-f63d6cc06414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:21.575753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:21.581151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff0e47-92f6-43ff-9de1-f63d6cc06414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:21.583082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff0e47-92f6-43ff-9de1-f63d6cc06414 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:21.717898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 106/206] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:21.724058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de81004-6aef-48d0-902d-6d2eb34ed207 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:21.840047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de81004-6aef-48d0-902d-6d2eb34ed207 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:21.878756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32d81b-9196-4425-9783-efe93914459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:21.879949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32d81b-9196-4425-9783-efe93914459d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:21.880387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32d81b-9196-4425-9783-efe93914459d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:21.880810 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32d81b-9196-4425-9783-efe93914459d None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:21.882107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32d81b-9196-4425-9783-efe93914459d None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:21.883765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32d81b-9196-4425-9783-efe93914459d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:21.885552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 101/207] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:21 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:21.894771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff0e47-92f6-43ff-9de1-f63d6cc06414 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:22.008770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:22.016447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff0e47-92f6-43ff-9de1-f63d6cc06414 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:22.054648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0498aa5-8f67-44c8-9eeb-9585f0f91bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:22.055259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0498aa5-8f67-44c8-9eeb-9585f0f91bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:22.055533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0498aa5-8f67-44c8-9eeb-9585f0f91bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:22.055802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0498aa5-8f67-44c8-9eeb-9585f0f91bb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:22.056525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0498aa5-8f67-44c8-9eeb-9585f0f91bb1 None admin] RBAC: Adding query filter params (role.id=853e41e951d14277bb9e21a8861bdbf9, scope.project.id=d78c4808482c4e74b6e8a3037b83de37, user.id=c5ff4a05676249e282fa93d97fcd6eb9) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:22.056828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0498aa5-8f67-44c8-9eeb-9585f0f91bb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:22.058364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0498aa5-8f67-44c8-9eeb-9585f0f91bb1 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:22.064715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 107/208] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:21 2022] GET /identity/v3/role_assignments?user.id=c5ff4a05676249e282fa93d97fcd6eb9&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=853e41e951d14277bb9e21a8861bdbf9 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:22.564311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7d13-3c0b-4215-ae75-5a4a6e58db77 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:22.564995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7d13-3c0b-4215-ae75-5a4a6e58db77 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:22.566093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff7d13-3c0b-4215-ae75-5a4a6e58db77 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:22.567956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 102/209] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:22.573306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71d999-333b-49ee-9994-0c7170012c98 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:22.574249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71d999-333b-49ee-9994-0c7170012c98 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:22.574716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71d999-333b-49ee-9994-0c7170012c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:22.601190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6b71d999-333b-49ee-9994-0c7170012c98 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:22.602808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6b71d999-333b-49ee-9994-0c7170012c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:22.680428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 108/210] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:22.730266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04896fbc-5a1a-4a14-a277-cdf0f1dfe5fb None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:22.730865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04896fbc-5a1a-4a14-a277-cdf0f1dfe5fb None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:22.731357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04896fbc-5a1a-4a14-a277-cdf0f1dfe5fb None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:22.733126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 103/211] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:22.738530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaede2-89e0-41a5-9116-7dfc9f855ef8 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:22.738811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaede2-89e0-41a5-9116-7dfc9f855ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:22.739080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eaede2-89e0-41a5-9116-7dfc9f855ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:22.757191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f7eaede2-89e0-41a5-9116-7dfc9f855ef8 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:22.758421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f7eaede2-89e0-41a5-9116-7dfc9f855ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:22.795454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d61fc-4ce0-4770-ad5f-c42d02e953a5 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:22.796046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d61fc-4ce0-4770-ad5f-c42d02e953a5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:22.796324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d61fc-4ce0-4770-ad5f-c42d02e953a5 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:22.797910 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 104/212] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:22.802864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e2671-d133-4def-a3a1-dc5325890ebe None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:22.803143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e2671-d133-4def-a3a1-dc5325890ebe None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:22.803414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e2671-d133-4def-a3a1-dc5325890ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:22.823558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-765e2671-d133-4def-a3a1-dc5325890ebe None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:22.824773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-765e2671-d133-4def-a3a1-dc5325890ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:22.834595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 109/213] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:22.894410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:22.906060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 105/214] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:22.978819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaede2-89e0-41a5-9116-7dfc9f855ef8 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:23.003547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:23.079521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eaede2-89e0-41a5-9116-7dfc9f855ef8 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:23.110449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e2671-d133-4def-a3a1-dc5325890ebe None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:23.113407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c105c-a042-48a6-ac8b-89562ddd5556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:23.114217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c105c-a042-48a6-ac8b-89562ddd5556 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.114495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c105c-a042-48a6-ac8b-89562ddd5556 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.114780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c105c-a042-48a6-ac8b-89562ddd5556 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.115950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c105c-a042-48a6-ac8b-89562ddd5556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:23.117487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c105c-a042-48a6-ac8b-89562ddd5556 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:23.119397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 110/215] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:23.124378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c105c-a042-48a6-ac8b-89562ddd5556 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:23.226873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e2671-d133-4def-a3a1-dc5325890ebe None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:23.239305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:23.251836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c105c-a042-48a6-ac8b-89562ddd5556 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:23.265041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72be7df-33d3-444f-b477-ccccb2b8de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:23.265728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72be7df-33d3-444f-b477-ccccb2b8de53 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.266015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72be7df-33d3-444f-b477-ccccb2b8de53 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.266287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72be7df-33d3-444f-b477-ccccb2b8de53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.280319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72be7df-33d3-444f-b477-ccccb2b8de53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-a72be7df-33d3-444f-b477-ccccb2b8de53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:23.285310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:23.286290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 106/216] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:23.290594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72be7df-33d3-444f-b477-ccccb2b8de53 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:23.295790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df465a2-f3a6-4226-83d4-dd1b53e26cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:23.296426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df465a2-f3a6-4226-83d4-dd1b53e26cc7 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.296702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df465a2-f3a6-4226-83d4-dd1b53e26cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.296974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df465a2-f3a6-4226-83d4-dd1b53e26cc7 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.297675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df465a2-f3a6-4226-83d4-dd1b53e26cc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:23.299161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df465a2-f3a6-4226-83d4-dd1b53e26cc7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:23.332302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 111/217] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:08:23 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:08:23.332302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df465a2-f3a6-4226-83d4-dd1b53e26cc7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:23.359924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:23.421620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72be7df-33d3-444f-b477-ccccb2b8de53 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:23.436838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df465a2-f3a6-4226-83d4-dd1b53e26cc7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:23.467857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff9324-46c2-4fa7-bf0b-8593fc2bd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:23.468868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff9324-46c2-4fa7-bf0b-8593fc2bd131 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.469393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff9324-46c2-4fa7-bf0b-8593fc2bd131 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.469833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff9324-46c2-4fa7-bf0b-8593fc2bd131 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.474514 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557b7e4-ce0f-4d02-aad3-21a237475748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:23.475137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557b7e4-ce0f-4d02-aad3-21a237475748 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.475413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557b7e4-ce0f-4d02-aad3-21a237475748 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.475685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557b7e4-ce0f-4d02-aad3-21a237475748 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.476359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557b7e4-ce0f-4d02-aad3-21a237475748 None admin] RBAC: Adding query filter params (name=member) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:23.476689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557b7e4-ce0f-4d02-aad3-21a237475748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:23.478092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557b7e4-ce0f-4d02-aad3-21a237475748 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:23.484343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff9324-46c2-4fa7-bf0b-8593fc2bd131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:23.487290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 112/218] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:23.488693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e28d5d-ac3f-4e41-afe3-ca0beea4f6ed None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.488978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e28d5d-ac3f-4e41-afe3-ca0beea4f6ed None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.489247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e28d5d-ac3f-4e41-afe3-ca0beea4f6ed None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.490751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 113/219] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-53ff9324-46c2-4fa7-bf0b-8593fc2bd131 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:23.490965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:23.492103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:23.493012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 107/220] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:23.494851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e28d5d-ac3f-4e41-afe3-ca0beea4f6ed None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:23.498368 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff9324-46c2-4fa7-bf0b-8593fc2bd131 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:23.510806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:23.626559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff9324-46c2-4fa7-bf0b-8593fc2bd131 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:23.627470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e28d5d-ac3f-4e41-afe3-ca0beea4f6ed None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:23.645932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:23.672701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee332e8d-eb7a-40bd-913a-9d114be2b82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:23.673340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee332e8d-eb7a-40bd-913a-9d114be2b82a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.673650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee332e8d-eb7a-40bd-913a-9d114be2b82a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.673922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee332e8d-eb7a-40bd-913a-9d114be2b82a None admin] PATH_INFO: `/v3/users/b3f4045ed2d34d59881c01764501f2fa` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.677447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26e351-27d3-4c3b-b205-858310ece25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:23.677606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26e351-27d3-4c3b-b205-858310ece25c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.677827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26e351-27d3-4c3b-b205-858310ece25c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.678095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26e351-27d3-4c3b-b205-858310ece25c None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.678791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26e351-27d3-4c3b-b205-858310ece25c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:23.679097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26e351-27d3-4c3b-b205-858310ece25c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:23.680482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26e351-27d3-4c3b-b205-858310ece25c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:23.690855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee332e8d-eb7a-40bd-913a-9d114be2b82a None admin] RBAC: Authorizing `identity:get_user(user_id=b3f4045ed2d34d59881c01764501f2fa)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:23.692357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee332e8d-eb7a-40bd-913a-9d114be2b82a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:23.694062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 114/221] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:23 2022] GET /identity/v3/users/b3f4045ed2d34d59881c01764501f2fa => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:23.695022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a50620-fe40-4c7c-b607-7e65cb5ec062 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.695296 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a50620-fe40-4c7c-b607-7e65cb5ec062 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.695559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a50620-fe40-4c7c-b607-7e65cb5ec062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.697563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 108/222] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:23.704633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26e351-27d3-4c3b-b205-858310ece25c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:23.715264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-72a50620-fe40-4c7c-b607-7e65cb5ec062 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:23.716218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-72a50620-fe40-4c7c-b607-7e65cb5ec062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:23.781951 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:23.789632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 115/223] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:23.790528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a50620-fe40-4c7c-b607-7e65cb5ec062 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:23.821001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26e351-27d3-4c3b-b205-858310ece25c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:23.869582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf47b-17db-4698-886a-295afe854e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:23.870214 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf47b-17db-4698-886a-295afe854e2a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.870491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf47b-17db-4698-886a-295afe854e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.870764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf47b-17db-4698-886a-295afe854e2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.922537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a50620-fe40-4c7c-b607-7e65cb5ec062 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:23.924356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:23.958709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cf47b-17db-4698-886a-295afe854e2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:23.958709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cf47b-17db-4698-886a-295afe854e2a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:23.976023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494a87d-2ad2-4558-8222-5a9b1c8039a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:23.977052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494a87d-2ad2-4558-8222-5a9b1c8039a4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:23.977555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494a87d-2ad2-4558-8222-5a9b1c8039a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:23.978031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494a87d-2ad2-4558-8222-5a9b1c8039a4 None admin] PATH_INFO: `/v3/users/0e83f81e6d634210aab861ccc61a0ab4` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:23.979233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494a87d-2ad2-4558-8222-5a9b1c8039a4 None admin] RBAC: Authorizing `identity:get_user(user_id=0e83f81e6d634210aab861ccc61a0ab4)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:23.980685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494a87d-2ad2-4558-8222-5a9b1c8039a4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:23.982508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 116/224] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:23 2022] GET /identity/v3/users/0e83f81e6d634210aab861ccc61a0ab4 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:23.988286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494a87d-2ad2-4558-8222-5a9b1c8039a4 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:24.055346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:24.099894 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494a87d-2ad2-4558-8222-5a9b1c8039a4 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:24.135297 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 109/225] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 20:08:24.136169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf47b-17db-4698-886a-295afe854e2a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:24.141387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74758812-75aa-4604-bee3-927d7c00aae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:24.142283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74758812-75aa-4604-bee3-927d7c00aae2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:24.142731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74758812-75aa-4604-bee3-927d7c00aae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:24.143727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74758812-75aa-4604-bee3-927d7c00aae2 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:24.145587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74758812-75aa-4604-bee3-927d7c00aae2 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:24.147699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74758812-75aa-4604-bee3-927d7c00aae2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:24.150532 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 117/226] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:23 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:24.151751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74758812-75aa-4604-bee3-927d7c00aae2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:24.231773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:24.259914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf47b-17db-4698-886a-295afe854e2a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:24.289960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74758812-75aa-4604-bee3-927d7c00aae2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:24.299988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f52c22-5392-4e10-b1e7-594b84ffd631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:24.300627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f52c22-5392-4e10-b1e7-594b84ffd631 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:24.300903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f52c22-5392-4e10-b1e7-594b84ffd631 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:24.301174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f52c22-5392-4e10-b1e7-594b84ffd631 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:24.314104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f52c22-5392-4e10-b1e7-594b84ffd631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-80f52c22-5392-4e10-b1e7-594b84ffd631 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:24.320181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:24.321042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:24.321818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 110/227] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:24.322458 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f52c22-5392-4e10-b1e7-594b84ffd631 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:24.335319 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:24.343340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760a751-3572-4ebd-a13c-1db7e47f2120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:24.344252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760a751-3572-4ebd-a13c-1db7e47f2120 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:24.344965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760a751-3572-4ebd-a13c-1db7e47f2120 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:24.345482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760a751-3572-4ebd-a13c-1db7e47f2120 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:24.348196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760a751-3572-4ebd-a13c-1db7e47f2120 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:24.350562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760a751-3572-4ebd-a13c-1db7e47f2120 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:24.352414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 118/228] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:24 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:24.353969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760a751-3572-4ebd-a13c-1db7e47f2120 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:24.447183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f52c22-5392-4e10-b1e7-594b84ffd631 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:24.486216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760a751-3572-4ebd-a13c-1db7e47f2120 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:24.491719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c562746-f8bb-4c43-98e4-d99e931ec0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:24.492490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c562746-f8bb-4c43-98e4-d99e931ec0a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:24.492986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c562746-f8bb-4c43-98e4-d99e931ec0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:24.493355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c562746-f8bb-4c43-98e4-d99e931ec0a4 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/users/0e83f81e6d634210aab861ccc61a0ab4/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:24.499710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c562746-f8bb-4c43-98e4-d99e931ec0a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9189f4b1cc4920b3079cdc0d358919, user_id=0e83f81e6d634210aab861ccc61a0ab4, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:24.502166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c562746-f8bb-4c43-98e4-d99e931ec0a4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:24.505750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:24.523370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 111/229] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:08:24 2022] PUT /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/users/0e83f81e6d634210aab861ccc61a0ab4/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:24.524013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c562746-f8bb-4c43-98e4-d99e931ec0a4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:24.550564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ac157-722d-4603-a8d2-7ec373b6fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:24.553282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ac157-722d-4603-a8d2-7ec373b6fdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:24.555019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ac157-722d-4603-a8d2-7ec373b6fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:24.555528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ac157-722d-4603-a8d2-7ec373b6fdb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:24.556423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ac157-722d-4603-a8d2-7ec373b6fdb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:24.559248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ac157-722d-4603-a8d2-7ec373b6fdb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:24.563001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ac157-722d-4603-a8d2-7ec373b6fdb3 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:24.582721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 119/230] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:24.594238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ac157-722d-4603-a8d2-7ec373b6fdb3 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:24.705672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:24.753079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c562746-f8bb-4c43-98e4-d99e931ec0a4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:24.765168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ac157-722d-4603-a8d2-7ec373b6fdb3 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:24.805948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09a0b3-5735-414e-bb6a-918242b3c0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:24.805948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a0b3-5735-414e-bb6a-918242b3c0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:24.805948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a0b3-5735-414e-bb6a-918242b3c0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:24.805948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a0b3-5735-414e-bb6a-918242b3c0a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:24.806331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09a0b3-5735-414e-bb6a-918242b3c0a1 None admin] RBAC: Adding query filter params (role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919, user.id=b3f4045ed2d34d59881c01764501f2fa) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:24.806331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09a0b3-5735-414e-bb6a-918242b3c0a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:24.807770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09a0b3-5735-414e-bb6a-918242b3c0a1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:24.814277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0b1bd-b050-4ca0-9f5b-23a6dc6ecdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:24.815767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0b1bd-b050-4ca0-9f5b-23a6dc6ecdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:24.815872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 112/231] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:24 2022] GET /identity/v3/role_assignments?user.id=b3f4045ed2d34d59881c01764501f2fa&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=84d6c85b41034cae925afeb8921381e8 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:24.816311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0b1bd-b050-4ca0-9f5b-23a6dc6ecdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:24.816582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0b1bd-b050-4ca0-9f5b-23a6dc6ecdd7 None admin] PATH_INFO: `/v3/users/0e83f81e6d634210aab861ccc61a0ab4` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:24.817739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0b1bd-b050-4ca0-9f5b-23a6dc6ecdd7 None admin] RBAC: Authorizing `identity:get_user(user_id=0e83f81e6d634210aab861ccc61a0ab4)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:24.819228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba0b1bd-b050-4ca0-9f5b-23a6dc6ecdd7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:24.821037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 120/232] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:24 2022] GET /identity/v3/users/0e83f81e6d634210aab861ccc61a0ab4 => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:24.826754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09a0b3-5735-414e-bb6a-918242b3c0a1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:24.895884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:24.943190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09a0b3-5735-414e-bb6a-918242b3c0a1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:24.979745 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8347d87-b89b-468e-a8b7-96e218d37644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:24.980396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8347d87-b89b-468e-a8b7-96e218d37644 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:24.980670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8347d87-b89b-468e-a8b7-96e218d37644 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:24.980937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8347d87-b89b-468e-a8b7-96e218d37644 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:24.982114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8347d87-b89b-468e-a8b7-96e218d37644 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:24.983596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8347d87-b89b-468e-a8b7-96e218d37644 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:24.989670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 113/233] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:24 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:24.992145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0b1bd-b050-4ca0-9f5b-23a6dc6ecdd7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:25.122710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0b1bd-b050-4ca0-9f5b-23a6dc6ecdd7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:25.162923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:25.174249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856dddbb-5ba1-4483-a65b-925e6c267bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:25.175492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dddbb-5ba1-4483-a65b-925e6c267bc6 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:25.175770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dddbb-5ba1-4483-a65b-925e6c267bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:25.176038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dddbb-5ba1-4483-a65b-925e6c267bc6 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/users/0e83f81e6d634210aab861ccc61a0ab4/roles/6a600852dc8442fdb2ffc3d663e274ff` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:25.178314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856dddbb-5ba1-4483-a65b-925e6c267bc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9189f4b1cc4920b3079cdc0d358919, user_id=0e83f81e6d634210aab861ccc61a0ab4, role_id=6a600852dc8442fdb2ffc3d663e274ff)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:25.179752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856dddbb-5ba1-4483-a65b-925e6c267bc6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:25.198026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 121/234] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:08:24 2022] PUT /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/users/0e83f81e6d634210aab861ccc61a0ab4/roles/6a600852dc8442fdb2ffc3d663e274ff => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:25.922412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d6c8-ae1e-4520-8c7b-bd4b6d37c9a1 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:25.922725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d6c8-ae1e-4520-8c7b-bd4b6d37c9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:25.922961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d6c8-ae1e-4520-8c7b-bd4b6d37c9a1 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:25.925142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 114/235] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:25.930771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bbe6a-b330-4b1f-83b7-be6e6e263c2c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:25.931232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bbe6a-b330-4b1f-83b7-be6e6e263c2c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:25.931305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bbe6a-b330-4b1f-83b7-be6e6e263c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:25.955876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-fb5bbe6a-b330-4b1f-83b7-be6e6e263c2c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:25.957069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fb5bbe6a-b330-4b1f-83b7-be6e6e263c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:26.047918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 122/236] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:26.121101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b84740-3922-4f99-80e5-e03f676df015 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:26.121623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b84740-3922-4f99-80e5-e03f676df015 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:26.121695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b84740-3922-4f99-80e5-e03f676df015 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:26.122990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 115/237] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:26.128268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04696c9d-09c0-46fe-8126-b30ef2eaaa45 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:26.128494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04696c9d-09c0-46fe-8126-b30ef2eaaa45 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:26.128762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04696c9d-09c0-46fe-8126-b30ef2eaaa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:26.154455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-04696c9d-09c0-46fe-8126-b30ef2eaaa45 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:26.155697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-04696c9d-09c0-46fe-8126-b30ef2eaaa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:26.235294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 123/238] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:26.422436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b84740-3922-4f99-80e5-e03f676df015 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:26.551601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b84740-3922-4f99-80e5-e03f676df015 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:26.590044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:26.599636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a758df-ee10-4d79-b0ff-b1d31e37493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:26.600268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a758df-ee10-4d79-b0ff-b1d31e37493c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:26.600598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a758df-ee10-4d79-b0ff-b1d31e37493c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:26.600871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a758df-ee10-4d79-b0ff-b1d31e37493c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:26.611162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a758df-ee10-4d79-b0ff-b1d31e37493c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-63a758df-ee10-4d79-b0ff-b1d31e37493c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:26.616835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:26.617713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:26.618460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 116/239] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:26.625184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04696c9d-09c0-46fe-8126-b30ef2eaaa45 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:26.626645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a758df-ee10-4d79-b0ff-b1d31e37493c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:26.683603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:26.749098 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04696c9d-09c0-46fe-8126-b30ef2eaaa45 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:26.787243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce99ff-eda1-4ced-a3f0-a18243c46700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:26.788223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce99ff-eda1-4ced-a3f0-a18243c46700 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:26.788692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce99ff-eda1-4ced-a3f0-a18243c46700 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:26.789154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce99ff-eda1-4ced-a3f0-a18243c46700 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:26.790101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce99ff-eda1-4ced-a3f0-a18243c46700 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:26.790734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce99ff-eda1-4ced-a3f0-a18243c46700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:26.792412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce99ff-eda1-4ced-a3f0-a18243c46700 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:26.802863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 124/240] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:26.810988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce99ff-eda1-4ced-a3f0-a18243c46700 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:26.825064 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a758df-ee10-4d79-b0ff-b1d31e37493c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:26.889549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a1aa-2661-4c95-8f6f-4c9dc1829caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:26.890203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a1aa-2661-4c95-8f6f-4c9dc1829caa None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:26.890488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a1aa-2661-4c95-8f6f-4c9dc1829caa None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:26.890933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799a1aa-2661-4c95-8f6f-4c9dc1829caa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:26.892056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799a1aa-2661-4c95-8f6f-4c9dc1829caa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:26.899016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799a1aa-2661-4c95-8f6f-4c9dc1829caa None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:26.900995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 117/241] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:26.905943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a1aa-2661-4c95-8f6f-4c9dc1829caa None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:27.001502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce99ff-eda1-4ced-a3f0-a18243c46700 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:27.017198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:27.029540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:27.074725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469785-a2ec-4795-a9ea-ef4c6bed20e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:27.075367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469785-a2ec-4795-a9ea-ef4c6bed20e1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.075640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469785-a2ec-4795-a9ea-ef4c6bed20e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.075906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469785-a2ec-4795-a9ea-ef4c6bed20e1 None admin] PATH_INFO: `/v3/users/e2d1b1c533744ec8a23f8c2e50173c52` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.085911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799a1aa-2661-4c95-8f6f-4c9dc1829caa None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:27.101443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2469785-a2ec-4795-a9ea-ef4c6bed20e1 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d1b1c533744ec8a23f8c2e50173c52)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:27.102946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2469785-a2ec-4795-a9ea-ef4c6bed20e1 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:27.104619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 125/242] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:26 2022] GET /identity/v3/users/e2d1b1c533744ec8a23f8c2e50173c52 => generated 297 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:27.110962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469785-a2ec-4795-a9ea-ef4c6bed20e1 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:27.137196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d72d7-3460-479f-ac67-beaeefd89204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:27.138510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d72d7-3460-479f-ac67-beaeefd89204 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.138995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d72d7-3460-479f-ac67-beaeefd89204 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.139463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d72d7-3460-479f-ac67-beaeefd89204 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.140848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d72d7-3460-479f-ac67-beaeefd89204 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:27.142714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d72d7-3460-479f-ac67-beaeefd89204 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:27.178628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 118/243] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:08:26 2022] POST /identity/v3/users => generated 293 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:08:27.270783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469785-a2ec-4795-a9ea-ef4c6bed20e1 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:27.299853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc333-841c-4f50-9e90-6cff94bfdc98 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.300474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc333-841c-4f50-9e90-6cff94bfdc98 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.300941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cc333-841c-4f50-9e90-6cff94bfdc98 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.302588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 119/244] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:08:27.308871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b8be1-28df-4979-b965-279254dea4f5 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.309544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b8be1-28df-4979-b965-279254dea4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.310019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b8be1-28df-4979-b965-279254dea4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.339768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9abb4b-c76a-4c39-8abe-1932d9cb3105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:27.340801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9abb4b-c76a-4c39-8abe-1932d9cb3105 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.341286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9abb4b-c76a-4c39-8abe-1932d9cb3105 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.341804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9abb4b-c76a-4c39-8abe-1932d9cb3105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.343948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8c5b8be1-28df-4979-b965-279254dea4f5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:27.348154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8c5b8be1-28df-4979-b965-279254dea4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:27.409923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:27.454683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9abb4b-c76a-4c39-8abe-1932d9cb3105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:27.457016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9abb4b-c76a-4c39-8abe-1932d9cb3105 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:27.482687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 120/245] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:27.483950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5febc5-80a8-4c8a-bee4-a83c8bc81db2 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.484235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5febc5-80a8-4c8a-bee4-a83c8bc81db2 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.484507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5febc5-80a8-4c8a-bee4-a83c8bc81db2 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.485806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 121/246] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:27.491183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a780d6-b682-4ab1-91b6-6377ce30560f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.491467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a780d6-b682-4ab1-91b6-6377ce30560f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.491743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a780d6-b682-4ab1-91b6-6377ce30560f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.515635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-83a780d6-b682-4ab1-91b6-6377ce30560f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:27.517812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-83a780d6-b682-4ab1-91b6-6377ce30560f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:27.561430 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:27.595068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 126/247] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 20:08:27.600641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9abb4b-c76a-4c39-8abe-1932d9cb3105 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:27.605724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 122/248] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:27.674308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:27.707961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9abb4b-c76a-4c39-8abe-1932d9cb3105 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:27.743875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5e291-26c0-4c1f-bb4f-9a2a6a9ef93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:27.744784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5e291-26c0-4c1f-bb4f-9a2a6a9ef93a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.745200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5e291-26c0-4c1f-bb4f-9a2a6a9ef93a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.745695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5e291-26c0-4c1f-bb4f-9a2a6a9ef93a None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.746949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5e291-26c0-4c1f-bb4f-9a2a6a9ef93a None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:27.748530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5e291-26c0-4c1f-bb4f-9a2a6a9ef93a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:27.750391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 127/249] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:27 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:27.756349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a780d6-b682-4ab1-91b6-6377ce30560f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:27.779190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5e291-26c0-4c1f-bb4f-9a2a6a9ef93a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:27.882685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a780d6-b682-4ab1-91b6-6377ce30560f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:27.909133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5e291-26c0-4c1f-bb4f-9a2a6a9ef93a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:27.927150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131b094-9fbc-40ef-863c-2547fda572c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:27.927862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131b094-9fbc-40ef-863c-2547fda572c4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.928204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131b094-9fbc-40ef-863c-2547fda572c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.928540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131b094-9fbc-40ef-863c-2547fda572c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.929446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131b094-9fbc-40ef-863c-2547fda572c4 None admin] RBAC: Adding query filter params (role.id=853e41e951d14277bb9e21a8861bdbf9, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919, user.id=e2d1b1c533744ec8a23f8c2e50173c52) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:27.929777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131b094-9fbc-40ef-863c-2547fda572c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:27.931233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131b094-9fbc-40ef-863c-2547fda572c4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:27.938235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 123/250] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:27 2022] GET /identity/v3/role_assignments?user.id=e2d1b1c533744ec8a23f8c2e50173c52&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=853e41e951d14277bb9e21a8861bdbf9 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:27.939294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131b094-9fbc-40ef-863c-2547fda572c4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:27.959930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e15d73-54f4-4d0a-8797-1c2420115505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:27.960589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e15d73-54f4-4d0a-8797-1c2420115505 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:27.960959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e15d73-54f4-4d0a-8797-1c2420115505 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:27.961175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e15d73-54f4-4d0a-8797-1c2420115505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:27.969671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:27.975391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e15d73-54f4-4d0a-8797-1c2420115505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-b7e15d73-54f4-4d0a-8797-1c2420115505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:27.981013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:27.981994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:27.982781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:27.983971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 128/251] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:27.992591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e15d73-54f4-4d0a-8797-1c2420115505 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:28.004693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:28.103579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131b094-9fbc-40ef-863c-2547fda572c4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:28.171759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e15d73-54f4-4d0a-8797-1c2420115505 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:28.174067 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f22cb-e043-420a-a4bf-78cc914cb017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:28.175339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f22cb-e043-420a-a4bf-78cc914cb017 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:28.175611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f22cb-e043-420a-a4bf-78cc914cb017 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:28.175876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f22cb-e043-420a-a4bf-78cc914cb017 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:28.189435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f22cb-e043-420a-a4bf-78cc914cb017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-8d0f22cb-e043-420a-a4bf-78cc914cb017 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:28.194759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:28.195624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:28.196386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 124/252] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:28.197035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8adbe-44e6-4719-bb61-2613a5d17253 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:28.197347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8adbe-44e6-4719-bb61-2613a5d17253 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:28.197637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8adbe-44e6-4719-bb61-2613a5d17253 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:28.201559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 125/253] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:28.204828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611ab7-229c-4bfa-85a0-19666f903bb5 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:28.205114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611ab7-229c-4bfa-85a0-19666f903bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:28.205391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611ab7-229c-4bfa-85a0-19666f903bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:28.216969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc61f4b-a94c-44d0-ae8b-7a82a4749f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:28.217871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc61f4b-a94c-44d0-ae8b-7a82a4749f7b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:28.218299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc61f4b-a94c-44d0-ae8b-7a82a4749f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:28.219212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc61f4b-a94c-44d0-ae8b-7a82a4749f7b None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:28.220328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc61f4b-a94c-44d0-ae8b-7a82a4749f7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:28.220809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc61f4b-a94c-44d0-ae8b-7a82a4749f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:28.222971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc61f4b-a94c-44d0-ae8b-7a82a4749f7b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:28.225026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-52611ab7-229c-4bfa-85a0-19666f903bb5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:28.226427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-52611ab7-229c-4bfa-85a0-19666f903bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:28.233082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 129/254] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:28.234208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc61f4b-a94c-44d0-ae8b-7a82a4749f7b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:28.306264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:28.347800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 126/255] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:28.348742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52611ab7-229c-4bfa-85a0-19666f903bb5 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:28.354832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc61f4b-a94c-44d0-ae8b-7a82a4749f7b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:28.398601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a973b8-30ac-41e3-ae7f-250b2dfa4582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:28.399541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a973b8-30ac-41e3-ae7f-250b2dfa4582 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:28.401070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a973b8-30ac-41e3-ae7f-250b2dfa4582 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:28.401545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a973b8-30ac-41e3-ae7f-250b2dfa4582 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:28.402458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a973b8-30ac-41e3-ae7f-250b2dfa4582 None admin] RBAC: Adding query filter params (name=member) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:28.402936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a973b8-30ac-41e3-ae7f-250b2dfa4582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:28.404571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a973b8-30ac-41e3-ae7f-250b2dfa4582 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:28.411119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:28.418784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 130/256] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:28.422971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a973b8-30ac-41e3-ae7f-250b2dfa4582 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:28.518114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52611ab7-229c-4bfa-85a0-19666f903bb5 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:28.554952 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65184bb-dfb3-429f-aa1a-e5c6a79c4df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:28.555595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65184bb-dfb3-429f-aa1a-e5c6a79c4df8 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:28.555872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65184bb-dfb3-429f-aa1a-e5c6a79c4df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:28.556141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65184bb-dfb3-429f-aa1a-e5c6a79c4df8 None admin] PATH_INFO: `/v3/users/0e83f81e6d634210aab861ccc61a0ab4` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:28.557315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65184bb-dfb3-429f-aa1a-e5c6a79c4df8 None admin] RBAC: Authorizing `identity:get_user(user_id=0e83f81e6d634210aab861ccc61a0ab4)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:28.558811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65184bb-dfb3-429f-aa1a-e5c6a79c4df8 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:28.565544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 127/257] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:28 2022] GET /identity/v3/users/0e83f81e6d634210aab861ccc61a0ab4 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:28.567239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65184bb-dfb3-429f-aa1a-e5c6a79c4df8 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:28.607608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a973b8-30ac-41e3-ae7f-250b2dfa4582 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:28.647438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:28.702312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da78969-8e05-4700-97dd-d26a75a101d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:28.702874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da78969-8e05-4700-97dd-d26a75a101d6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:28.703148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da78969-8e05-4700-97dd-d26a75a101d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:28.703416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da78969-8e05-4700-97dd-d26a75a101d6 None admin] PATH_INFO: `/v3/users/b3f4045ed2d34d59881c01764501f2fa` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:28.704580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da78969-8e05-4700-97dd-d26a75a101d6 None admin] RBAC: Authorizing `identity:get_user(user_id=b3f4045ed2d34d59881c01764501f2fa)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:28.706628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da78969-8e05-4700-97dd-d26a75a101d6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:28.707177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65184bb-dfb3-429f-aa1a-e5c6a79c4df8 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:28.708524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 131/258] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:28 2022] GET /identity/v3/users/b3f4045ed2d34d59881c01764501f2fa => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:28.708788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:28.713937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da78969-8e05-4700-97dd-d26a75a101d6 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:28.743063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf9ec0-9323-4ec4-8edb-04f8b5b9a2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:28.743914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf9ec0-9323-4ec4-8edb-04f8b5b9a2cd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:28.744339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf9ec0-9323-4ec4-8edb-04f8b5b9a2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:28.744712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf9ec0-9323-4ec4-8edb-04f8b5b9a2cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:28.821604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf9ec0-9323-4ec4-8edb-04f8b5b9a2cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:28.823253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf9ec0-9323-4ec4-8edb-04f8b5b9a2cd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:28.853890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da78969-8e05-4700-97dd-d26a75a101d6 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:28.900245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d261f03-75c9-4226-a6be-f8ebb96a3664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:28.901166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d261f03-75c9-4226-a6be-f8ebb96a3664 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:28.901625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d261f03-75c9-4226-a6be-f8ebb96a3664 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:28.902123 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d261f03-75c9-4226-a6be-f8ebb96a3664 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:28.903445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d261f03-75c9-4226-a6be-f8ebb96a3664 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:28.905096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d261f03-75c9-4226-a6be-f8ebb96a3664 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:28.907305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 132/259] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:28 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:28.908385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d261f03-75c9-4226-a6be-f8ebb96a3664 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:28.961966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:28.978384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 128/260] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:28.979503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf9ec0-9323-4ec4-8edb-04f8b5b9a2cd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:29.023382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:29.123533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d261f03-75c9-4226-a6be-f8ebb96a3664 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:29.150549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf9ec0-9323-4ec4-8edb-04f8b5b9a2cd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:29.186322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d63a6-88d9-4a8d-8bdb-1574c91ea073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:29.187749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d63a6-88d9-4a8d-8bdb-1574c91ea073 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:29.188272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d63a6-88d9-4a8d-8bdb-1574c91ea073 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:29.189022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d63a6-88d9-4a8d-8bdb-1574c91ea073 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/users/b3f4045ed2d34d59881c01764501f2fa/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:29.191817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d63a6-88d9-4a8d-8bdb-1574c91ea073 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9189f4b1cc4920b3079cdc0d358919, user_id=b3f4045ed2d34d59881c01764501f2fa, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:29.193476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d63a6-88d9-4a8d-8bdb-1574c91ea073 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:29.202263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be308a-9820-43d5-91cf-32bbda4fa30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:29.202909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be308a-9820-43d5-91cf-32bbda4fa30c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:29.203195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be308a-9820-43d5-91cf-32bbda4fa30c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:29.203538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be308a-9820-43d5-91cf-32bbda4fa30c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:29.211655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 129/261] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:08:28 2022] PUT /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/users/b3f4045ed2d34d59881c01764501f2fa/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:29.212885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d63a6-88d9-4a8d-8bdb-1574c91ea073 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:29.227525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be308a-9820-43d5-91cf-32bbda4fa30c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-86be308a-9820-43d5-91cf-32bbda4fa30c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:29.237505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:29.238495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:29.239276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 133/262] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:29.248545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be308a-9820-43d5-91cf-32bbda4fa30c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:29.331013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:29.433394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d63a6-88d9-4a8d-8bdb-1574c91ea073 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:29.473945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b2f5-ff3c-41ad-9b48-0f5be24fbe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:29.474530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b2f5-ff3c-41ad-9b48-0f5be24fbe19 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:29.474804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b2f5-ff3c-41ad-9b48-0f5be24fbe19 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:29.475071 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b2f5-ff3c-41ad-9b48-0f5be24fbe19 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:29.476187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3b2f5-ff3c-41ad-9b48-0f5be24fbe19 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:29.477735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3b2f5-ff3c-41ad-9b48-0f5be24fbe19 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:29.479358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 130/263] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:29 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:29.485237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b2f5-ff3c-41ad-9b48-0f5be24fbe19 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:29.531692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be308a-9820-43d5-91cf-32bbda4fa30c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:29.553883 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:29.574311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906b258-b155-42e9-ab07-957adc194572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:29.575506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906b258-b155-42e9-ab07-957adc194572 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:29.575960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906b258-b155-42e9-ab07-957adc194572 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:29.576592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906b258-b155-42e9-ab07-957adc194572 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:29.578293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906b258-b155-42e9-ab07-957adc194572 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:29.578770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906b258-b155-42e9-ab07-957adc194572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:29.580333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906b258-b155-42e9-ab07-957adc194572 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:29.598916 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 134/264] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:29.604903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906b258-b155-42e9-ab07-957adc194572 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:29.631425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b2f5-ff3c-41ad-9b48-0f5be24fbe19 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:29.673774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:29.699467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c9b29-25ea-477f-8ba9-59f2be89677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:29.700416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c9b29-25ea-477f-8ba9-59f2be89677e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:29.700889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c9b29-25ea-477f-8ba9-59f2be89677e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:29.701374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c9b29-25ea-477f-8ba9-59f2be89677e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:29.706955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c9b29-25ea-477f-8ba9-59f2be89677e None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919, user.id=0e83f81e6d634210aab861ccc61a0ab4) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:29.707500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c9b29-25ea-477f-8ba9-59f2be89677e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:29.709391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c9b29-25ea-477f-8ba9-59f2be89677e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:29.722167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 131/265] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:29 2022] GET /identity/v3/role_assignments?user.id=0e83f81e6d634210aab861ccc61a0ab4&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=36331ec562ff42f883fc883363bc4c8d => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:29.777904 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906b258-b155-42e9-ab07-957adc194572 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:29.820979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d2869-af9c-45f1-b46c-0f5ea96f94de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:29.821657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d2869-af9c-45f1-b46c-0f5ea96f94de None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:29.821939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d2869-af9c-45f1-b46c-0f5ea96f94de None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:29.822214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d2869-af9c-45f1-b46c-0f5ea96f94de None admin] PATH_INFO: `/v3/users/0e83f81e6d634210aab861ccc61a0ab4` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:29.824112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d2869-af9c-45f1-b46c-0f5ea96f94de None admin] RBAC: Authorizing `identity:get_user(user_id=0e83f81e6d634210aab861ccc61a0ab4)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:29.825669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d2869-af9c-45f1-b46c-0f5ea96f94de None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:29.827729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 135/266] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:29 2022] GET /identity/v3/users/0e83f81e6d634210aab861ccc61a0ab4 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:29.833555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c9b29-25ea-477f-8ba9-59f2be89677e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:29.968918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c9b29-25ea-477f-8ba9-59f2be89677e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:30.008391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44449ac5-3c7f-4ae5-8efc-c41d0346896f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:30.009394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44449ac5-3c7f-4ae5-8efc-c41d0346896f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:30.009890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44449ac5-3c7f-4ae5-8efc-c41d0346896f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:30.010353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44449ac5-3c7f-4ae5-8efc-c41d0346896f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:30.069269 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:30.090317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44449ac5-3c7f-4ae5-8efc-c41d0346896f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:30.092212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44449ac5-3c7f-4ae5-8efc-c41d0346896f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:30.225404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 132/267] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:30.231520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d2869-af9c-45f1-b46c-0f5ea96f94de None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:30.298231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:30.359037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d2869-af9c-45f1-b46c-0f5ea96f94de None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:30.401021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f75c6-91db-4103-b92d-ddb5255c1931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:30.401691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f75c6-91db-4103-b92d-ddb5255c1931 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:30.401973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f75c6-91db-4103-b92d-ddb5255c1931 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:30.402245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f75c6-91db-4103-b92d-ddb5255c1931 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:30.404354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f75c6-91db-4103-b92d-ddb5255c1931 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:30.405995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f75c6-91db-4103-b92d-ddb5255c1931 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:30.407659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 136/268] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:30 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:30.413726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44449ac5-3c7f-4ae5-8efc-c41d0346896f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:30.545813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:30.565772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44449ac5-3c7f-4ae5-8efc-c41d0346896f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:30.618727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7393f9-01d6-4b7c-aa12-cbe2b23df15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:30.619275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7393f9-01d6-4b7c-aa12-cbe2b23df15c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:30.619550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7393f9-01d6-4b7c-aa12-cbe2b23df15c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:30.619819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7393f9-01d6-4b7c-aa12-cbe2b23df15c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:30.620570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7393f9-01d6-4b7c-aa12-cbe2b23df15c None admin] RBAC: Adding query filter params (role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919, user.id=0e83f81e6d634210aab861ccc61a0ab4) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:30.620877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7393f9-01d6-4b7c-aa12-cbe2b23df15c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:30.622540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7393f9-01d6-4b7c-aa12-cbe2b23df15c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:30.635059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 133/269] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:30 2022] GET /identity/v3/role_assignments?user.id=0e83f81e6d634210aab861ccc61a0ab4&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:30.751371 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49951a8-8afb-4bb0-acf8-7284cb4169ae None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:30.751705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49951a8-8afb-4bb0-acf8-7284cb4169ae None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:30.751920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49951a8-8afb-4bb0-acf8-7284cb4169ae None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:30.753740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 137/270] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:30.760007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f8054-20fc-44cb-b445-dfee1e617517 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:30.760353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f8054-20fc-44cb-b445-dfee1e617517 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:30.760553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f8054-20fc-44cb-b445-dfee1e617517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:30.790738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-896f8054-20fc-44cb-b445-dfee1e617517 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:30.791992 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-896f8054-20fc-44cb-b445-dfee1e617517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:30.865270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f1c6-08a4-4596-be6f-8532aeec74c3 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:30.865616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f1c6-08a4-4596-be6f-8532aeec74c3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:30.866005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1f1c6-08a4-4596-be6f-8532aeec74c3 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:30.867339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 138/271] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:30.872901 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e079-b8bc-4d01-855d-bc098393fec2 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:30.873147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e079-b8bc-4d01-855d-bc098393fec2 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:30.873366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e079-b8bc-4d01-855d-bc098393fec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:30.895593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6608e079-b8bc-4d01-855d-bc098393fec2 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:30.897076 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6608e079-b8bc-4d01-855d-bc098393fec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:30.897477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 134/272] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:30.967800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 139/273] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:31.190164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f8054-20fc-44cb-b445-dfee1e617517 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:31.229863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f059d1-3874-437a-88c0-7ec58f1d4661 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:31.230135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f059d1-3874-437a-88c0-7ec58f1d4661 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:31.230655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f059d1-3874-437a-88c0-7ec58f1d4661 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:31.232179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 140/274] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:31.238313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03968ddb-b579-4f9c-8703-19d3abf77a36 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:31.238822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03968ddb-b579-4f9c-8703-19d3abf77a36 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:31.239278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03968ddb-b579-4f9c-8703-19d3abf77a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:31.260089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-03968ddb-b579-4f9c-8703-19d3abf77a36 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:31.261273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-03968ddb-b579-4f9c-8703-19d3abf77a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:31.275371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:31.299237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f8054-20fc-44cb-b445-dfee1e617517 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:31.307504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:31.335204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 141/275] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:31.341702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03968ddb-b579-4f9c-8703-19d3abf77a36 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:31.344292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d30e5-01f5-4c8e-9fda-3a1d6de4c7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:31.345097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d30e5-01f5-4c8e-9fda-3a1d6de4c7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:31.345538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d30e5-01f5-4c8e-9fda-3a1d6de4c7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:31.346468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d30e5-01f5-4c8e-9fda-3a1d6de4c7b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:31.348118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d30e5-01f5-4c8e-9fda-3a1d6de4c7b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:31.350278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3d30e5-01f5-4c8e-9fda-3a1d6de4c7b5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:31.353032 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 135/276] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:31.358119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d30e5-01f5-4c8e-9fda-3a1d6de4c7b5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:31.443192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03968ddb-b579-4f9c-8703-19d3abf77a36 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:31.461137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d30e5-01f5-4c8e-9fda-3a1d6de4c7b5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:31.479683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a795a-2b73-4774-bda3-01db1f03ad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:31.480316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a795a-2b73-4774-bda3-01db1f03ad91 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:31.480643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a795a-2b73-4774-bda3-01db1f03ad91 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:31.480919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a795a-2b73-4774-bda3-01db1f03ad91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:31.492042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a795a-2b73-4774-bda3-01db1f03ad91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:31.496780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f65fcd-0a57-47ee-8a84-978d0ab0d6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-a54a795a-2b73-4774-bda3-01db1f03ad91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:31.497372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:31.498497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:31.499378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:31.500279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f65fcd-0a57-47ee-8a84-978d0ab0d6bf None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:31.500688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f65fcd-0a57-47ee-8a84-978d0ab0d6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:31.501114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f65fcd-0a57-47ee-8a84-978d0ab0d6bf None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:31.501912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f65fcd-0a57-47ee-8a84-978d0ab0d6bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:31.503977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 142/277] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:31.504289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f65fcd-0a57-47ee-8a84-978d0ab0d6bf None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:31.505244 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a795a-2b73-4774-bda3-01db1f03ad91 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:31.537769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 136/278] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:08:31 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:08:31.544250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:31.616190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f65fcd-0a57-47ee-8a84-978d0ab0d6bf None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:31.619578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:31.620475 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a795a-2b73-4774-bda3-01db1f03ad91 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:31.667812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0beff1-a132-4f5a-bd0a-8b0c115e64df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:31.668423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0beff1-a132-4f5a-bd0a-8b0c115e64df None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:31.668698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0beff1-a132-4f5a-bd0a-8b0c115e64df None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:31.668967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0beff1-a132-4f5a-bd0a-8b0c115e64df None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:31.669672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0beff1-a132-4f5a-bd0a-8b0c115e64df None admin] RBAC: Adding query filter params (name=member) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:31.669973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0beff1-a132-4f5a-bd0a-8b0c115e64df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:31.671781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0beff1-a132-4f5a-bd0a-8b0c115e64df None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:31.681859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 143/279] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:31.688369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0beff1-a132-4f5a-bd0a-8b0c115e64df None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:31.760760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f65fcd-0a57-47ee-8a84-978d0ab0d6bf None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:31.804707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35d6e1-1a12-4139-b48d-89f0daeb97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:31.805350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35d6e1-1a12-4139-b48d-89f0daeb97de None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:31.805672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35d6e1-1a12-4139-b48d-89f0daeb97de None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:31.805965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35d6e1-1a12-4139-b48d-89f0daeb97de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:31.819839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35d6e1-1a12-4139-b48d-89f0daeb97de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:31.820445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:31.823915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0beff1-a132-4f5a-bd0a-8b0c115e64df None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-ff35d6e1-1a12-4139-b48d-89f0daeb97de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:31.825329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:31.826293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:31.827051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:31.828086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 137/280] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:31.834667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35d6e1-1a12-4139-b48d-89f0daeb97de None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:31.868044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f902f-d30f-43f7-b4ea-60ff9d276440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:31.869088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f902f-d30f-43f7-b4ea-60ff9d276440 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:31.869433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f902f-d30f-43f7-b4ea-60ff9d276440 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:31.869841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f902f-d30f-43f7-b4ea-60ff9d276440 None admin] PATH_INFO: `/v3/users/0a71186d91224b2497bab4ea191a38b8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:31.888645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f902f-d30f-43f7-b4ea-60ff9d276440 None admin] RBAC: Authorizing `identity:get_user(user_id=0a71186d91224b2497bab4ea191a38b8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:31.890452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f902f-d30f-43f7-b4ea-60ff9d276440 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:31.893444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 144/281] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:31 2022] GET /identity/v3/users/0a71186d91224b2497bab4ea191a38b8 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:31.898760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f902f-d30f-43f7-b4ea-60ff9d276440 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:31.941293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:31.997075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35d6e1-1a12-4139-b48d-89f0daeb97de None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:32.034899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f902f-d30f-43f7-b4ea-60ff9d276440 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:32.040832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa56ff1-57f6-42cf-a442-82fd120e63f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:32.041456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa56ff1-57f6-42cf-a442-82fd120e63f6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:32.041763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa56ff1-57f6-42cf-a442-82fd120e63f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:32.042033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa56ff1-57f6-42cf-a442-82fd120e63f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:32.042693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa56ff1-57f6-42cf-a442-82fd120e63f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:32.042994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa56ff1-57f6-42cf-a442-82fd120e63f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:32.044376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa56ff1-57f6-42cf-a442-82fd120e63f6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:32.053639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 138/282] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:32.054881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a789d-7a5a-45c9-95e3-8a4cf2a0d565 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:32.055162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a789d-7a5a-45c9-95e3-8a4cf2a0d565 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:32.055429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a789d-7a5a-45c9-95e3-8a4cf2a0d565 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:32.056752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 139/283] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:32.063734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a789d-7a5a-45c9-95e3-8a4cf2a0d565 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:32.083637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6be43d-e8eb-4255-be14-fad9d05a7130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:32.084789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be43d-e8eb-4255-be14-fad9d05a7130 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:32.085241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be43d-e8eb-4255-be14-fad9d05a7130 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:32.085951 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be43d-e8eb-4255-be14-fad9d05a7130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:32.087260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6be43d-e8eb-4255-be14-fad9d05a7130 None admin] RBAC: Adding query filter params (role.id=84d6c85b41034cae925afeb8921381e8, user.id=0a71186d91224b2497bab4ea191a38b8, scope.system=all) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:32.087769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6be43d-e8eb-4255-be14-fad9d05a7130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:32.090012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6be43d-e8eb-4255-be14-fad9d05a7130 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:32.097860 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 145/284] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Oct 5 20:08:31 2022] GET /identity/v3/role_assignments?user.id=0a71186d91224b2497bab4ea191a38b8&scope.system=all&role.id=84d6c85b41034cae925afeb8921381e8 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:32.099670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f9195-9fe3-4dea-972e-4a1b7fcf88cc None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:32.100588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f9195-9fe3-4dea-972e-4a1b7fcf88cc None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:32.101199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f9195-9fe3-4dea-972e-4a1b7fcf88cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:32.128726 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-fa0f9195-9fe3-4dea-972e-4a1b7fcf88cc None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:32.130827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fa0f9195-9fe3-4dea-972e-4a1b7fcf88cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:32.153947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:32.215601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a789d-7a5a-45c9-95e3-8a4cf2a0d565 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:32.264226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 146/285] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:32.268725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:32.296344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baf621-f110-4502-8194-8d2e246fe367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:32.296946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baf621-f110-4502-8194-8d2e246fe367 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:32.297156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baf621-f110-4502-8194-8d2e246fe367 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:32.297390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baf621-f110-4502-8194-8d2e246fe367 None admin] PATH_INFO: `/v3/users/e2d1b1c533744ec8a23f8c2e50173c52` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:32.302277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56baf621-f110-4502-8194-8d2e246fe367 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d1b1c533744ec8a23f8c2e50173c52)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:32.303815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56baf621-f110-4502-8194-8d2e246fe367 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:32.305456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 140/286] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:32 2022] GET /identity/v3/users/e2d1b1c533744ec8a23f8c2e50173c52 => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:32.311539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f9195-9fe3-4dea-972e-4a1b7fcf88cc None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:32.421045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f9195-9fe3-4dea-972e-4a1b7fcf88cc None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:32.462748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3ab1c-d1ab-48fd-8caa-b6da456bc725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:32.463724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3ab1c-d1ab-48fd-8caa-b6da456bc725 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:32.464191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3ab1c-d1ab-48fd-8caa-b6da456bc725 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:32.464651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3ab1c-d1ab-48fd-8caa-b6da456bc725 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:32.465993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3ab1c-d1ab-48fd-8caa-b6da456bc725 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:32.467632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3ab1c-d1ab-48fd-8caa-b6da456bc725 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:32.469457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 147/287] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:32 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:32.476029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baf621-f110-4502-8194-8d2e246fe367 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:32.600233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3ab1c-d1ab-48fd-8caa-b6da456bc725 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:32.624836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baf621-f110-4502-8194-8d2e246fe367 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:32.661872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:32.662177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97117cd8-198b-4260-8ade-28c7a5123de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:32.662764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97117cd8-198b-4260-8ade-28c7a5123de3 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:32.663041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97117cd8-198b-4260-8ade-28c7a5123de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:32.663313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97117cd8-198b-4260-8ade-28c7a5123de3 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/users/e2d1b1c533744ec8a23f8c2e50173c52/roles/853e41e951d14277bb9e21a8861bdbf9` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:32.668275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97117cd8-198b-4260-8ade-28c7a5123de3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9189f4b1cc4920b3079cdc0d358919, user_id=e2d1b1c533744ec8a23f8c2e50173c52, role_id=853e41e951d14277bb9e21a8861bdbf9)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:32.669808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97117cd8-198b-4260-8ade-28c7a5123de3 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:32.696598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 141/288] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:08:32 2022] PUT /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/users/e2d1b1c533744ec8a23f8c2e50173c52/roles/853e41e951d14277bb9e21a8861bdbf9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:32.766905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3ab1c-d1ab-48fd-8caa-b6da456bc725 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:32.807381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75858587-5bdc-463d-ae49-0108b997f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:32.808141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75858587-5bdc-463d-ae49-0108b997f318 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:32.808422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75858587-5bdc-463d-ae49-0108b997f318 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:32.808687 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75858587-5bdc-463d-ae49-0108b997f318 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:32.821236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75858587-5bdc-463d-ae49-0108b997f318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-75858587-5bdc-463d-ae49-0108b997f318 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:32.827715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:32.828631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:32.829388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 148/289] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:32.836447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97117cd8-198b-4260-8ade-28c7a5123de3 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:32.914113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:32.965197 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97117cd8-198b-4260-8ade-28c7a5123de3 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:33.003503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7cd4b-e46f-4d90-827d-4784af488646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:33.004174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7cd4b-e46f-4d90-827d-4784af488646 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:33.004455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7cd4b-e46f-4d90-827d-4784af488646 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:33.004724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7cd4b-e46f-4d90-827d-4784af488646 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:33.005385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7cd4b-e46f-4d90-827d-4784af488646 None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:33.005753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7cd4b-e46f-4d90-827d-4784af488646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:33.007181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb7cd4b-e46f-4d90-827d-4784af488646 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:33.015929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 142/290] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:33.022290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75858587-5bdc-463d-ae49-0108b997f318 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:33.124933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:33.151415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75858587-5bdc-463d-ae49-0108b997f318 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:33.197569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4839e5-f753-4697-b0e7-78d520c01716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:33.198211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4839e5-f753-4697-b0e7-78d520c01716 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:33.198485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4839e5-f753-4697-b0e7-78d520c01716 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:33.198749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4839e5-f753-4697-b0e7-78d520c01716 None admin] PATH_INFO: `/v3/users/b3f4045ed2d34d59881c01764501f2fa` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:33.199870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4839e5-f753-4697-b0e7-78d520c01716 None admin] RBAC: Authorizing `identity:get_user(user_id=b3f4045ed2d34d59881c01764501f2fa)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:33.201315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4839e5-f753-4697-b0e7-78d520c01716 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:33.204191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 149/291] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:33 2022] GET /identity/v3/users/b3f4045ed2d34d59881c01764501f2fa => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:33.209041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7cd4b-e46f-4d90-827d-4784af488646 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:33.312398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7cd4b-e46f-4d90-827d-4784af488646 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:33.326962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:33.348582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622dcf7-8ec6-4fb8-9443-2b5a6bfe30f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:33.349194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622dcf7-8ec6-4fb8-9443-2b5a6bfe30f4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:33.349475 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622dcf7-8ec6-4fb8-9443-2b5a6bfe30f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:33.349785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622dcf7-8ec6-4fb8-9443-2b5a6bfe30f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:33.422965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622dcf7-8ec6-4fb8-9443-2b5a6bfe30f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:33.424601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622dcf7-8ec6-4fb8-9443-2b5a6bfe30f4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:33.532200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 143/292] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:33.538531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4839e5-f753-4697-b0e7-78d520c01716 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:33.690674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4839e5-f753-4697-b0e7-78d520c01716 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:33.743910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97c326-7340-4a5d-aef7-de2e017edc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:33.744543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97c326-7340-4a5d-aef7-de2e017edc3b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:33.744818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97c326-7340-4a5d-aef7-de2e017edc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:33.745082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97c326-7340-4a5d-aef7-de2e017edc3b None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:33.750205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97c326-7340-4a5d-aef7-de2e017edc3b None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:33.751656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97c326-7340-4a5d-aef7-de2e017edc3b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:33.753622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 150/293] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:33 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:33.759121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622dcf7-8ec6-4fb8-9443-2b5a6bfe30f4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:33.785450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:33.936110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622dcf7-8ec6-4fb8-9443-2b5a6bfe30f4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:33.985180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c5691-1843-47f1-b75e-778f2dab09d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:33.985733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c5691-1843-47f1-b75e-778f2dab09d7 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:33.986008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c5691-1843-47f1-b75e-778f2dab09d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:33.986278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c5691-1843-47f1-b75e-778f2dab09d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:33.987043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c5691-1843-47f1-b75e-778f2dab09d7 None admin] RBAC: Adding query filter params (role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919, user.id=b3f4045ed2d34d59881c01764501f2fa) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:33.987343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c5691-1843-47f1-b75e-778f2dab09d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:33.988774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c5691-1843-47f1-b75e-778f2dab09d7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:34.000138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 144/294] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:33 2022] GET /identity/v3/role_assignments?user.id=b3f4045ed2d34d59881c01764501f2fa&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=84d6c85b41034cae925afeb8921381e8 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:34.257866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaf4f7-7852-4c10-a096-90f5ef056d9f None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:34.258046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaf4f7-7852-4c10-a096-90f5ef056d9f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:34.258323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaf4f7-7852-4c10-a096-90f5ef056d9f None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:34.259897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 151/295] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:34.265277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbbd49-269e-4da2-a3c3-a6f2bf19e722 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:34.265574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbbd49-269e-4da2-a3c3-a6f2bf19e722 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:34.265849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbbd49-269e-4da2-a3c3-a6f2bf19e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:34.285852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-02dbbd49-269e-4da2-a3c3-a6f2bf19e722 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:34.287035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-02dbbd49-269e-4da2-a3c3-a6f2bf19e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:34.362481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 145/296] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:34.647335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c928-2980-46eb-8835-a0891bbad506 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:34.647875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c928-2980-46eb-8835-a0891bbad506 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:34.647875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630c928-2980-46eb-8835-a0891bbad506 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:34.649452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 152/297] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:34.654769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f33fd-9f8f-407a-87a3-df683a3db96f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:34.655051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f33fd-9f8f-407a-87a3-df683a3db96f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:34.655323 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f33fd-9f8f-407a-87a3-df683a3db96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:34.661301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c928-2980-46eb-8835-a0891bbad506 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:34.680114 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:34.681053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-279f33fd-9f8f-407a-87a3-df683a3db96f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:34.682435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-279f33fd-9f8f-407a-87a3-df683a3db96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:34.687018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:34.765605 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 146/298] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:34.788236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630c928-2980-46eb-8835-a0891bbad506 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:34.824383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081ea01-824f-4a1c-b837-85cdb203cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:34.825331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081ea01-824f-4a1c-b837-85cdb203cd56 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:34.825841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081ea01-824f-4a1c-b837-85cdb203cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:34.826390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081ea01-824f-4a1c-b837-85cdb203cd56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:34.838213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081ea01-824f-4a1c-b837-85cdb203cd56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-8081ea01-824f-4a1c-b837-85cdb203cd56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:34.843484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:34.844305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:34.845062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:34.848323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 153/299] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:34.854335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f33fd-9f8f-407a-87a3-df683a3db96f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:34.969856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f33fd-9f8f-407a-87a3-df683a3db96f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:35.006309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e1075-bffa-4610-81e1-97f5632c3759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:35.006936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e1075-bffa-4610-81e1-97f5632c3759 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.007216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e1075-bffa-4610-81e1-97f5632c3759 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.007489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e1075-bffa-4610-81e1-97f5632c3759 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.008155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e1075-bffa-4610-81e1-97f5632c3759 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:35.008457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e1075-bffa-4610-81e1-97f5632c3759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:35.010125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e1075-bffa-4610-81e1-97f5632c3759 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:35.015510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:35.020562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 147/300] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:35.023940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9fa2-9ebc-4e34-a0e4-4a5b5471a23a None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.024320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9fa2-9ebc-4e34-a0e4-4a5b5471a23a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.024601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc9fa2-9ebc-4e34-a0e4-4a5b5471a23a None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.026000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 154/301] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:35.030782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e1075-bffa-4610-81e1-97f5632c3759 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:35.031092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86edca-b3fa-4b10-8a90-1c37d72632e4 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.031373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86edca-b3fa-4b10-8a90-1c37d72632e4 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.031639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86edca-b3fa-4b10-8a90-1c37d72632e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.049844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5d86edca-b3fa-4b10-8a90-1c37d72632e4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:35.051032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5d86edca-b3fa-4b10-8a90-1c37d72632e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:35.122365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 155/302] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:35.123279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86edca-b3fa-4b10-8a90-1c37d72632e4 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:35.130585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:35.146973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e1075-bffa-4610-81e1-97f5632c3759 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:35.184763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fea977-ef38-41db-a2a2-08318a260c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:35.185836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fea977-ef38-41db-a2a2-08318a260c41 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.186385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fea977-ef38-41db-a2a2-08318a260c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.186949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fea977-ef38-41db-a2a2-08318a260c41 None admin] PATH_INFO: `/v3/users/2ce6ac3f7afa4246b7d7258ed3a4b0c7` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.204391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fea977-ef38-41db-a2a2-08318a260c41 None admin] RBAC: Authorizing `identity:get_user(user_id=2ce6ac3f7afa4246b7d7258ed3a4b0c7)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:35.206865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fea977-ef38-41db-a2a2-08318a260c41 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:35.209176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 148/303] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:35 2022] GET /identity/v3/users/2ce6ac3f7afa4246b7d7258ed3a4b0c7 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:35.221609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fea977-ef38-41db-a2a2-08318a260c41 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:35.257067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86edca-b3fa-4b10-8a90-1c37d72632e4 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:35.293504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:35.295171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7a3a6-b7e7-4d14-93a9-c3ff61e2a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:35.296152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7a3a6-b7e7-4d14-93a9-c3ff61e2a113 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.296152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7a3a6-b7e7-4d14-93a9-c3ff61e2a113 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.296370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7a3a6-b7e7-4d14-93a9-c3ff61e2a113 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.297707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7a3a6-b7e7-4d14-93a9-c3ff61e2a113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:35.299830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7a3a6-b7e7-4d14-93a9-c3ff61e2a113 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:35.301397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 156/304] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:35.307059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7a3a6-b7e7-4d14-93a9-c3ff61e2a113 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:35.348796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fea977-ef38-41db-a2a2-08318a260c41 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:35.380042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:35.387133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbb394-baba-4c5b-85d7-9e8e92cdb09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:35.387744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbb394-baba-4c5b-85d7-9e8e92cdb09b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.388021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbb394-baba-4c5b-85d7-9e8e92cdb09b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.388295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbb394-baba-4c5b-85d7-9e8e92cdb09b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.389023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbb394-baba-4c5b-85d7-9e8e92cdb09b None admin] RBAC: Adding query filter params (role.id=853e41e951d14277bb9e21a8861bdbf9, user.id=2ce6ac3f7afa4246b7d7258ed3a4b0c7, scope.system=all) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:35.389329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbb394-baba-4c5b-85d7-9e8e92cdb09b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:35.390808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbb394-baba-4c5b-85d7-9e8e92cdb09b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:35.396189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 149/305] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Oct 5 20:08:35 2022] GET /identity/v3/role_assignments?user.id=2ce6ac3f7afa4246b7d7258ed3a4b0c7&scope.system=all&role.id=853e41e951d14277bb9e21a8861bdbf9 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:35.397356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b2a9f-dad2-4c30-8c55-85d6d7f523df None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.397669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b2a9f-dad2-4c30-8c55-85d6d7f523df None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.397949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b2a9f-dad2-4c30-8c55-85d6d7f523df None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.399194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 150/306] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:35.401816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b2a9f-dad2-4c30-8c55-85d6d7f523df None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:35.418523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7a3a6-b7e7-4d14-93a9-c3ff61e2a113 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:35.455160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120ce4-a4af-4fb7-8dde-13d7d7acd9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:35.456374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120ce4-a4af-4fb7-8dde-13d7d7acd9ff None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.456646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120ce4-a4af-4fb7-8dde-13d7d7acd9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.456972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120ce4-a4af-4fb7-8dde-13d7d7acd9ff None admin] PATH_INFO: `/v3/groups` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.457653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28120ce4-a4af-4fb7-8dde-13d7d7acd9ff None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:35.459321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28120ce4-a4af-4fb7-8dde-13d7d7acd9ff None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:35.479318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 157/307] 10.222.0.23 () {60 vars in 1134 bytes} [Wed Oct 5 20:08:35 2022] POST /identity/v3/groups => generated 229 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:08:35.480856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0600e6c-b15e-4b76-b554-14b73dddd766 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.481139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0600e6c-b15e-4b76-b554-14b73dddd766 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.481459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0600e6c-b15e-4b76-b554-14b73dddd766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.505480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e0600e6c-b15e-4b76-b554-14b73dddd766 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:35.507696 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e0600e6c-b15e-4b76-b554-14b73dddd766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:35.529326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b2a9f-dad2-4c30-8c55-85d6d7f523df None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:35.557975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:35.572939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0901e-4b36-4656-9dac-9baa61a66fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:35.573922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0901e-4b36-4656-9dac-9baa61a66fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.574390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0901e-4b36-4656-9dac-9baa61a66fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.574797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0901e-4b36-4656-9dac-9baa61a66fa4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.588588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0901e-4b36-4656-9dac-9baa61a66fa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-23d0901e-4b36-4656-9dac-9baa61a66fa4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:35.594274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:35.595401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:35.596871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:35.605549 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 158/308] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:35.605746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 151/309] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:35.607773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0600e6c-b15e-4b76-b554-14b73dddd766 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:35.672892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:35.716795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0600e6c-b15e-4b76-b554-14b73dddd766 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:35.751819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ee391-834c-4147-a76a-ced0a4e705f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:35.752886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ee391-834c-4147-a76a-ced0a4e705f2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.753161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ee391-834c-4147-a76a-ced0a4e705f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.753426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ee391-834c-4147-a76a-ced0a4e705f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.754115 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ee391-834c-4147-a76a-ced0a4e705f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:35.754412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ee391-834c-4147-a76a-ced0a4e705f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:35.755775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ee391-834c-4147-a76a-ced0a4e705f2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:35.764444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 159/310] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:35.770645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0901e-4b36-4656-9dac-9baa61a66fa4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:35.887055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0901e-4b36-4656-9dac-9baa61a66fa4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:35.890529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ee391-834c-4147-a76a-ced0a4e705f2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:35.931100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d003a31-a2d2-4aae-ac14-070c08264b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:35.931728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d003a31-a2d2-4aae-ac14-070c08264b42 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:35.932000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d003a31-a2d2-4aae-ac14-070c08264b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:35.932266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d003a31-a2d2-4aae-ac14-070c08264b42 None admin] PATH_INFO: `/v3/users/0a71186d91224b2497bab4ea191a38b8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:35.933980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d003a31-a2d2-4aae-ac14-070c08264b42 None admin] RBAC: Authorizing `identity:get_user(user_id=0a71186d91224b2497bab4ea191a38b8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:35.935406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d003a31-a2d2-4aae-ac14-070c08264b42 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:35.942096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 152/311] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:35 2022] GET /identity/v3/users/0a71186d91224b2497bab4ea191a38b8 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:35.945094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d003a31-a2d2-4aae-ac14-070c08264b42 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:36.016809 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:36.019949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ee391-834c-4147-a76a-ced0a4e705f2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:36.053339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:36.070039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66561-2a1c-4601-9316-2e689bc51d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:36.071555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66561-2a1c-4601-9316-2e689bc51d40 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:36.072125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66561-2a1c-4601-9316-2e689bc51d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:36.072588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e66561-2a1c-4601-9316-2e689bc51d40 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:36.073786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d003a31-a2d2-4aae-ac14-070c08264b42 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:36.086161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e66561-2a1c-4601-9316-2e689bc51d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-77e66561-2a1c-4601-9316-2e689bc51d40 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:36.093771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:36.094603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:36.095390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:36.103231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 160/312] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:36.104861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66561-2a1c-4601-9316-2e689bc51d40 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:36.116990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ece58f-e19a-47f5-ba57-ce02fe891e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:36.117736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ece58f-e19a-47f5-ba57-ce02fe891e1d None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:36.118013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ece58f-e19a-47f5-ba57-ce02fe891e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:36.118285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ece58f-e19a-47f5-ba57-ce02fe891e1d None admin] PATH_INFO: `/v3/system/users/0a71186d91224b2497bab4ea191a38b8/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:36.119779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ece58f-e19a-47f5-ba57-ce02fe891e1d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0a71186d91224b2497bab4ea191a38b8, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:36.121255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ece58f-e19a-47f5-ba57-ce02fe891e1d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:36.141540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 153/313] 10.222.0.23 () {58 vars in 1334 bytes} [Wed Oct 5 20:08:35 2022] PUT /identity/v3/system/users/0a71186d91224b2497bab4ea191a38b8/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:36.235567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e66561-2a1c-4601-9316-2e689bc51d40 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:36.273460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a84fb0-451b-4423-9b40-f2cf5d539882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:36.274479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a84fb0-451b-4423-9b40-f2cf5d539882 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:36.274967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a84fb0-451b-4423-9b40-f2cf5d539882 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:36.275432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a84fb0-451b-4423-9b40-f2cf5d539882 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:36.276306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a84fb0-451b-4423-9b40-f2cf5d539882 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:36.276811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a84fb0-451b-4423-9b40-f2cf5d539882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:36.278450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a84fb0-451b-4423-9b40-f2cf5d539882 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:36.289129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 161/314] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:36.295476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ece58f-e19a-47f5-ba57-ce02fe891e1d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:36.407022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ece58f-e19a-47f5-ba57-ce02fe891e1d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:36.422159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:36.444705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cf2f0-348a-41be-adce-dab4f29c43a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:36.445677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cf2f0-348a-41be-adce-dab4f29c43a1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:36.446147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cf2f0-348a-41be-adce-dab4f29c43a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:36.446604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cf2f0-348a-41be-adce-dab4f29c43a1 None admin] PATH_INFO: `/v3/users/e2d1b1c533744ec8a23f8c2e50173c52` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:36.447905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cf2f0-348a-41be-adce-dab4f29c43a1 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d1b1c533744ec8a23f8c2e50173c52)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:36.449540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cf2f0-348a-41be-adce-dab4f29c43a1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:36.451607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 154/315] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:36 2022] GET /identity/v3/users/e2d1b1c533744ec8a23f8c2e50173c52 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:36.458131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a84fb0-451b-4423-9b40-f2cf5d539882 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:36.508655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:36.565216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a84fb0-451b-4423-9b40-f2cf5d539882 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:36.600143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787c1c6-e879-419d-b328-aa9207ca3ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:36.601074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787c1c6-e879-419d-b328-aa9207ca3ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:36.601585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787c1c6-e879-419d-b328-aa9207ca3ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:36.602038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787c1c6-e879-419d-b328-aa9207ca3ea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:36.671538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787c1c6-e879-419d-b328-aa9207ca3ea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:36.672987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787c1c6-e879-419d-b328-aa9207ca3ea7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:36.719835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:36.775392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 162/316] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:36.780980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cf2f0-348a-41be-adce-dab4f29c43a1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:36.893437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cf2f0-348a-41be-adce-dab4f29c43a1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:36.927535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa05fe-6896-443c-a933-750cb01fd063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:36.928162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa05fe-6896-443c-a933-750cb01fd063 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:36.928437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa05fe-6896-443c-a933-750cb01fd063 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:36.928708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa05fe-6896-443c-a933-750cb01fd063 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:36.929839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa05fe-6896-443c-a933-750cb01fd063 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:36.931279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa05fe-6896-443c-a933-750cb01fd063 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:36.933239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 155/317] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:36 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:36.939391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787c1c6-e879-419d-b328-aa9207ca3ea7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:37.043391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787c1c6-e879-419d-b328-aa9207ca3ea7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:37.077976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd66383-6107-415a-aa1d-a9728f5028a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:37.078948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd66383-6107-415a-aa1d-a9728f5028a7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:37.079440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd66383-6107-415a-aa1d-a9728f5028a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:37.079926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd66383-6107-415a-aa1d-a9728f5028a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:37.081425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd66383-6107-415a-aa1d-a9728f5028a7 None admin] RBAC: Adding query filter params (role.id=853e41e951d14277bb9e21a8861bdbf9, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919, user.id=e2d1b1c533744ec8a23f8c2e50173c52) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:37.081941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd66383-6107-415a-aa1d-a9728f5028a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:37.083541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd66383-6107-415a-aa1d-a9728f5028a7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:37.089892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 163/318] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:08:36 2022] GET /identity/v3/role_assignments?user.id=e2d1b1c533744ec8a23f8c2e50173c52&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=853e41e951d14277bb9e21a8861bdbf9 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:37.605725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15e2bb-2761-4646-bd6d-72ec4850cf72 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:37.606233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15e2bb-2761-4646-bd6d-72ec4850cf72 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:37.606305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be15e2bb-2761-4646-bd6d-72ec4850cf72 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:37.607813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 156/319] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:37.613325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed918afe-e8c7-4e54-91c9-5fe4224cc0db None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:37.613631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed918afe-e8c7-4e54-91c9-5fe4224cc0db None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:37.613901 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed918afe-e8c7-4e54-91c9-5fe4224cc0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:37.632481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ed918afe-e8c7-4e54-91c9-5fe4224cc0db None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:37.633696 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ed918afe-e8c7-4e54-91c9-5fe4224cc0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:37.636492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:37.700762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 164/320] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:37.815618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cec7ed-a08b-42b8-8e1d-0462b7362faf None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:37.815954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cec7ed-a08b-42b8-8e1d-0462b7362faf None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:37.816168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cec7ed-a08b-42b8-8e1d-0462b7362faf None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:37.817644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 157/321] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:37.822630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a89dd-7ce8-4e42-92b2-c4dc30f3a784 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:37.822921 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a89dd-7ce8-4e42-92b2-c4dc30f3a784 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:37.823193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a89dd-7ce8-4e42-92b2-c4dc30f3a784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:37.841297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-573a89dd-7ce8-4e42-92b2-c4dc30f3a784 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:37.842505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-573a89dd-7ce8-4e42-92b2-c4dc30f3a784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:37.915590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 165/322] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:37.965292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cec7ed-a08b-42b8-8e1d-0462b7362faf None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:38.015570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:38.067327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cec7ed-a08b-42b8-8e1d-0462b7362faf None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:38.099847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e75bba-9cf8-46b8-9ed8-b4818fa867cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:38.100483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e75bba-9cf8-46b8-9ed8-b4818fa867cb None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:38.100752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e75bba-9cf8-46b8-9ed8-b4818fa867cb None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:38.101020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e75bba-9cf8-46b8-9ed8-b4818fa867cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:38.110830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e75bba-9cf8-46b8-9ed8-b4818fa867cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-81e75bba-9cf8-46b8-9ed8-b4818fa867cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:38.116554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:38.117361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:38.118144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:38.121848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 158/323] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:38.123363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a89dd-7ce8-4e42-92b2-c4dc30f3a784 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:38.183995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e75bba-9cf8-46b8-9ed8-b4818fa867cb None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:38.192873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:38.227679 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a89dd-7ce8-4e42-92b2-c4dc30f3a784 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:38.267685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc6623-2578-46d3-b5fa-b19fa40d1d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:38.269189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc6623-2578-46d3-b5fa-b19fa40d1d6a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:38.269466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc6623-2578-46d3-b5fa-b19fa40d1d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:38.269810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc6623-2578-46d3-b5fa-b19fa40d1d6a None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:38.270405 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc6623-2578-46d3-b5fa-b19fa40d1d6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:38.270702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc6623-2578-46d3-b5fa-b19fa40d1d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:38.272182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc6623-2578-46d3-b5fa-b19fa40d1d6a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:38.284938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 166/324] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:38.286195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc6623-2578-46d3-b5fa-b19fa40d1d6a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:38.297336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e75bba-9cf8-46b8-9ed8-b4818fa867cb None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:38.310593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:38.339056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534665f5-8117-4778-b877-0ff7794e7b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:38.339808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534665f5-8117-4778-b877-0ff7794e7b61 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:38.340087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534665f5-8117-4778-b877-0ff7794e7b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:38.340351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534665f5-8117-4778-b877-0ff7794e7b61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:38.341444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534665f5-8117-4778-b877-0ff7794e7b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:38.344216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534665f5-8117-4778-b877-0ff7794e7b61 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:38.345817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 159/325] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:38.351181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534665f5-8117-4778-b877-0ff7794e7b61 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:38.396919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc6623-2578-46d3-b5fa-b19fa40d1d6a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:38.425221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:38.430377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de6056-adf3-4e49-bdda-cea697916dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:38.431004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6056-adf3-4e49-bdda-cea697916dbc None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:38.431281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6056-adf3-4e49-bdda-cea697916dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:38.431552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6056-adf3-4e49-bdda-cea697916dbc None admin] PATH_INFO: `/v3/users/2ce6ac3f7afa4246b7d7258ed3a4b0c7` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:38.432675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de6056-adf3-4e49-bdda-cea697916dbc None admin] RBAC: Authorizing `identity:get_user(user_id=2ce6ac3f7afa4246b7d7258ed3a4b0c7)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:38.434210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de6056-adf3-4e49-bdda-cea697916dbc None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:38.435712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 167/326] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:38 2022] GET /identity/v3/users/2ce6ac3f7afa4246b7d7258ed3a4b0c7 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:38.436816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f526c-0cfb-448f-8457-1c6c527e6aa5 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:38.437093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f526c-0cfb-448f-8457-1c6c527e6aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:38.437365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f526c-0cfb-448f-8457-1c6c527e6aa5 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:38.438828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 168/327] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:38.445372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f526c-0cfb-448f-8457-1c6c527e6aa5 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:38.457546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534665f5-8117-4778-b877-0ff7794e7b61 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:38.491018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aa92a-f732-4451-b809-3d34cb931aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:38.491621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aa92a-f732-4451-b809-3d34cb931aba None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:38.491899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aa92a-f732-4451-b809-3d34cb931aba None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:38.492171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aa92a-f732-4451-b809-3d34cb931aba None admin] PATH_INFO: `/v3/groups` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:38.492790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772aa92a-f732-4451-b809-3d34cb931aba None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:38.494246 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772aa92a-f732-4451-b809-3d34cb931aba None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:38.509766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 160/328] 10.222.0.23 () {60 vars in 1134 bytes} [Wed Oct 5 20:08:38 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:08:38.511078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adeca2-ffa8-4d10-8427-0ad66ecaecec None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:38.511361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adeca2-ffa8-4d10-8427-0ad66ecaecec None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:38.511631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adeca2-ffa8-4d10-8427-0ad66ecaecec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:38.534557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c3adeca2-ffa8-4d10-8427-0ad66ecaecec None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:38.535887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c3adeca2-ffa8-4d10-8427-0ad66ecaecec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:38.562842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f526c-0cfb-448f-8457-1c6c527e6aa5 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:38.572170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:38.603259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e32eb1-1b8c-4f16-9fda-ad9790b1904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:38.603924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e32eb1-1b8c-4f16-9fda-ad9790b1904a None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:38.604196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e32eb1-1b8c-4f16-9fda-ad9790b1904a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:38.604466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e32eb1-1b8c-4f16-9fda-ad9790b1904a None admin] PATH_INFO: `/v3/system/users/2ce6ac3f7afa4246b7d7258ed3a4b0c7/roles/853e41e951d14277bb9e21a8861bdbf9` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:38.606009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e32eb1-1b8c-4f16-9fda-ad9790b1904a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2ce6ac3f7afa4246b7d7258ed3a4b0c7, role_id=853e41e951d14277bb9e21a8861bdbf9)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:38.607414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e32eb1-1b8c-4f16-9fda-ad9790b1904a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:38.617956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 169/329] 10.222.0.23 () {58 vars in 1334 bytes} [Wed Oct 5 20:08:38 2022] PUT /identity/v3/system/users/2ce6ac3f7afa4246b7d7258ed3a4b0c7/roles/853e41e951d14277bb9e21a8861bdbf9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:38.619151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 161/330] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:38.892408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e32eb1-1b8c-4f16-9fda-ad9790b1904a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:38.956938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:39.003336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e32eb1-1b8c-4f16-9fda-ad9790b1904a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:39.042299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472708d-556d-470f-95c1-f67aff872108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:39.042936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472708d-556d-470f-95c1-f67aff872108 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:39.043213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472708d-556d-470f-95c1-f67aff872108 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:39.043480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472708d-556d-470f-95c1-f67aff872108 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:39.054922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472708d-556d-470f-95c1-f67aff872108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-0472708d-556d-470f-95c1-f67aff872108 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:39.060324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:39.061197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:39.062008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 170/331] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:39.068106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adeca2-ffa8-4d10-8427-0ad66ecaecec None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:39.173099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adeca2-ffa8-4d10-8427-0ad66ecaecec None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:39.209141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d9717-fe93-45ba-8479-5bff6649c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:39.209794 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d9717-fe93-45ba-8479-5bff6649c808 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:39.210074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d9717-fe93-45ba-8479-5bff6649c808 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:39.210345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d9717-fe93-45ba-8479-5bff6649c808 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:39.211023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d9717-fe93-45ba-8479-5bff6649c808 None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:39.211324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d9717-fe93-45ba-8479-5bff6649c808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:39.212715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d9717-fe93-45ba-8479-5bff6649c808 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:39.222358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 162/332] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:39.228002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472708d-556d-470f-95c1-f67aff872108 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:39.339586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472708d-556d-470f-95c1-f67aff872108 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:39.366235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:39.378201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf22331-bb66-490e-8e3d-707d2467ae1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:39.378840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf22331-bb66-490e-8e3d-707d2467ae1e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:39.379112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf22331-bb66-490e-8e3d-707d2467ae1e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:39.379376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf22331-bb66-490e-8e3d-707d2467ae1e None admin] PATH_INFO: `/v3/users/0a71186d91224b2497bab4ea191a38b8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:39.380565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf22331-bb66-490e-8e3d-707d2467ae1e None admin] RBAC: Authorizing `identity:get_user(user_id=0a71186d91224b2497bab4ea191a38b8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:39.382061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf22331-bb66-490e-8e3d-707d2467ae1e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:39.383625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 171/333] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:39 2022] GET /identity/v3/users/0a71186d91224b2497bab4ea191a38b8 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:39.389695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d9717-fe93-45ba-8479-5bff6649c808 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:39.429721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:39.511296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d9717-fe93-45ba-8479-5bff6649c808 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:39.548046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d949e6-327a-4f9d-a640-e418264a3bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:39.548652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d949e6-327a-4f9d-a640-e418264a3bcf None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:39.548927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d949e6-327a-4f9d-a640-e418264a3bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:39.549854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d949e6-327a-4f9d-a640-e418264a3bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:39.550578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d949e6-327a-4f9d-a640-e418264a3bcf None admin] RBAC: Adding query filter params (role.id=84d6c85b41034cae925afeb8921381e8, user.id=0a71186d91224b2497bab4ea191a38b8, scope.system=all) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:39.550879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d949e6-327a-4f9d-a640-e418264a3bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:39.552384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d949e6-327a-4f9d-a640-e418264a3bcf None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:39.566523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 163/334] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Oct 5 20:08:39 2022] GET /identity/v3/role_assignments?user.id=0a71186d91224b2497bab4ea191a38b8&scope.system=all&role.id=84d6c85b41034cae925afeb8921381e8 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:40.988733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5bfa1-106a-4cf4-adf0-bacea73eddd7 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:40.989273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5bfa1-106a-4cf4-adf0-bacea73eddd7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:40.989273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5bfa1-106a-4cf4-adf0-bacea73eddd7 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:40.990944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 172/335] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:40.996702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a277e-6431-4bc7-9486-9b6f8100c08a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:40.997139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a277e-6431-4bc7-9486-9b6f8100c08a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:40.997230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a277e-6431-4bc7-9486-9b6f8100c08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:41.021044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-316a277e-6431-4bc7-9486-9b6f8100c08a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:41.022255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-316a277e-6431-4bc7-9486-9b6f8100c08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:41.113279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 164/336] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:41.198438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3609b5-3ced-406f-9330-e1670645874e None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:41.198724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3609b5-3ced-406f-9330-e1670645874e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:41.198915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3609b5-3ced-406f-9330-e1670645874e None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:41.200495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 173/337] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:41.214193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec37934-2134-4c18-a537-3f3f93fe868f None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:41.214466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec37934-2134-4c18-a537-3f3f93fe868f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:41.214740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec37934-2134-4c18-a537-3f3f93fe868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:41.218803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50459ccb-ed7c-4c77-b322-21615d5769d3 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:41.219081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50459ccb-ed7c-4c77-b322-21615d5769d3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:41.219349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50459ccb-ed7c-4c77-b322-21615d5769d3 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:41.222067 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 165/338] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:41.227393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5bb3a-2db6-4804-8847-94a1dd0a644d None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:41.227659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5bb3a-2db6-4804-8847-94a1dd0a644d None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:41.227925 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5bb3a-2db6-4804-8847-94a1dd0a644d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:41.254611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7ec37934-2134-4c18-a537-3f3f93fe868f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:41.255836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7ec37934-2134-4c18-a537-3f3f93fe868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:41.265145 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-15b5bb3a-2db6-4804-8847-94a1dd0a644d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:41.266364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-15b5bb3a-2db6-4804-8847-94a1dd0a644d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:41.289890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:41.411689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 174/339] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:41.419275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 166/340] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:41.537717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec37934-2134-4c18-a537-3f3f93fe868f None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:41.652525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec37934-2134-4c18-a537-3f3f93fe868f None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:41.681361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:41.698037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efbcdc-a2b6-4522-889e-86b48792b6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:41.698739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efbcdc-a2b6-4522-889e-86b48792b6dc None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:41.699014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efbcdc-a2b6-4522-889e-86b48792b6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:41.699389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efbcdc-a2b6-4522-889e-86b48792b6dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:41.720193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efbcdc-a2b6-4522-889e-86b48792b6dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-74efbcdc-a2b6-4522-889e-86b48792b6dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:41.725235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:41.726091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:41.727054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 175/341] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:41.732868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5bb3a-2db6-4804-8847-94a1dd0a644d None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:41.797606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efbcdc-a2b6-4522-889e-86b48792b6dc None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:41.882956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5bb3a-2db6-4804-8847-94a1dd0a644d None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:41.931625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523f397-fd80-4b02-96d5-1ec65b2f05a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:41.932244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523f397-fd80-4b02-96d5-1ec65b2f05a6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:41.932516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523f397-fd80-4b02-96d5-1ec65b2f05a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:41.932779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523f397-fd80-4b02-96d5-1ec65b2f05a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:41.933443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523f397-fd80-4b02-96d5-1ec65b2f05a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:41.933769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523f397-fd80-4b02-96d5-1ec65b2f05a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:41.935243 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523f397-fd80-4b02-96d5-1ec65b2f05a6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:41.950639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 167/342] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:41.951331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523f397-fd80-4b02-96d5-1ec65b2f05a6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:41.988446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:42.031043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efbcdc-a2b6-4522-889e-86b48792b6dc None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:42.098582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b0716-b131-4034-8ba7-673ee4e11b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:42.099217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b0716-b131-4034-8ba7-673ee4e11b56 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:42.099490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b0716-b131-4034-8ba7-673ee4e11b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:42.099757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b0716-b131-4034-8ba7-673ee4e11b56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:42.137746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b0716-b131-4034-8ba7-673ee4e11b56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-c41b0716-b131-4034-8ba7-673ee4e11b56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:42.142566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:42.143972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 176/343] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:42.145312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b0716-b131-4034-8ba7-673ee4e11b56 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:42.201402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523f397-fd80-4b02-96d5-1ec65b2f05a6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:42.233913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:42.275922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f3615-a3a8-4e9b-9598-bdd85c01a2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:42.276565 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f3615-a3a8-4e9b-9598-bdd85c01a2be None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:42.276839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f3615-a3a8-4e9b-9598-bdd85c01a2be None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:42.277105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f3615-a3a8-4e9b-9598-bdd85c01a2be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:42.302655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f3615-a3a8-4e9b-9598-bdd85c01a2be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-e55f3615-a3a8-4e9b-9598-bdd85c01a2be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:42.309500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:42.310415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 20:08:42.311185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:42.314596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 168/344] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:42.316763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f3615-a3a8-4e9b-9598-bdd85c01a2be None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:42.347897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b0716-b131-4034-8ba7-673ee4e11b56 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:42.419691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147b92f-bda3-4b5a-a28f-c540313a4cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:42.420225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b92f-bda3-4b5a-a28f-c540313a4cae None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:42.420494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b92f-bda3-4b5a-a28f-c540313a4cae None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:42.420754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b92f-bda3-4b5a-a28f-c540313a4cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:42.457949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:42.491644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f3615-a3a8-4e9b-9598-bdd85c01a2be None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:42.552974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e8c5f-4e14-40a4-9361-049e91c37a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:42.553609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e8c5f-4e14-40a4-9361-049e91c37a07 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:42.553885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e8c5f-4e14-40a4-9361-049e91c37a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:42.554152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e8c5f-4e14-40a4-9361-049e91c37a07 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:42.554815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e8c5f-4e14-40a4-9361-049e91c37a07 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:42.555151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e8c5f-4e14-40a4-9361-049e91c37a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:42.556531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e8c5f-4e14-40a4-9361-049e91c37a07 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:42.568084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147b92f-bda3-4b5a-a28f-c540313a4cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:42.569566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147b92f-bda3-4b5a-a28f-c540313a4cae None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:42.576068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 169/345] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:42.576923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e8c5f-4e14-40a4-9361-049e91c37a07 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:42.649968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:42.732171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 177/346] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:42.733383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e8c5f-4e14-40a4-9361-049e91c37a07 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:42.733746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147b92f-bda3-4b5a-a28f-c540313a4cae None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:42.777087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:42.785095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ba814-ec30-46c8-88ab-ae3aebf3df35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:42.785773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ba814-ec30-46c8-88ab-ae3aebf3df35 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:42.786049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ba814-ec30-46c8-88ab-ae3aebf3df35 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:42.786319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ba814-ec30-46c8-88ab-ae3aebf3df35 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:42.786988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ba814-ec30-46c8-88ab-ae3aebf3df35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:42.787281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ba814-ec30-46c8-88ab-ae3aebf3df35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:42.788658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ba814-ec30-46c8-88ab-ae3aebf3df35 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:42.801399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 170/347] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:42.805538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609ff18-84b7-4c7c-bd65-14edfeceb06e None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:42.805538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609ff18-84b7-4c7c-bd65-14edfeceb06e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:42.805538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609ff18-84b7-4c7c-bd65-14edfeceb06e None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:42.806318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 171/348] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:42.807436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609ff18-84b7-4c7c-bd65-14edfeceb06e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:42.888314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147b92f-bda3-4b5a-a28f-c540313a4cae None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:42.927099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab65a97-7089-4d55-89ca-9615890cb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:42.927717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65a97-7089-4d55-89ca-9615890cb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:42.927993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65a97-7089-4d55-89ca-9615890cb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:42.928260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65a97-7089-4d55-89ca-9615890cb2b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:42.941738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:42.958146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609ff18-84b7-4c7c-bd65-14edfeceb06e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:43.007050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ce144-5285-456d-8231-6a144ea0ac00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:43.008226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ce144-5285-456d-8231-6a144ea0ac00 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.008722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ce144-5285-456d-8231-6a144ea0ac00 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.009210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ce144-5285-456d-8231-6a144ea0ac00 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.010250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab65a97-7089-4d55-89ca-9615890cb2b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:43.010985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ce144-5285-456d-8231-6a144ea0ac00 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:43.011736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab65a97-7089-4d55-89ca-9615890cb2b7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:43.013018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ce144-5285-456d-8231-6a144ea0ac00 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:43.016615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 172/349] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:42 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:43.018238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb87aa1-512a-452c-95f9-46801e14852a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.018741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb87aa1-512a-452c-95f9-46801e14852a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.019220 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb87aa1-512a-452c-95f9-46801e14852a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.041268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7bb87aa1-512a-452c-95f9-46801e14852a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:43.043120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7bb87aa1-512a-452c-95f9-46801e14852a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:43.114824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:43.130385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 173/350] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:43.131447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb87aa1-512a-452c-95f9-46801e14852a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:43.136756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 178/351] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:43.136756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab65a97-7089-4d55-89ca-9615890cb2b7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:43.205918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:43.259955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb87aa1-512a-452c-95f9-46801e14852a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:43.269758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab65a97-7089-4d55-89ca-9615890cb2b7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:43.311653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf6f98-434f-4c5b-a9df-838ac22bedd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:43.312294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf6f98-434f-4c5b-a9df-838ac22bedd2 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.312570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf6f98-434f-4c5b-a9df-838ac22bedd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.312837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf6f98-434f-4c5b-a9df-838ac22bedd2 None admin] PATH_INFO: `/v3/users/2ce6ac3f7afa4246b7d7258ed3a4b0c7` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.314741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf6f98-434f-4c5b-a9df-838ac22bedd2 None admin] RBAC: Authorizing `identity:get_user(user_id=2ce6ac3f7afa4246b7d7258ed3a4b0c7)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:43.316250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf6f98-434f-4c5b-a9df-838ac22bedd2 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:43.318093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 174/352] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:08:43 2022] GET /identity/v3/users/2ce6ac3f7afa4246b7d7258ed3a4b0c7 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:43.319212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55694e17-05aa-456a-ad0b-61a6c6281cec None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.319493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55694e17-05aa-456a-ad0b-61a6c6281cec None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.319763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55694e17-05aa-456a-ad0b-61a6c6281cec None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.320616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e066a9-e974-433a-8e1c-e70666e66308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:43.321182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 175/353] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:43.321780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e066a9-e974-433a-8e1c-e70666e66308 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.321904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55694e17-05aa-456a-ad0b-61a6c6281cec None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:43.322494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e066a9-e974-433a-8e1c-e70666e66308 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.322973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e066a9-e974-433a-8e1c-e70666e66308 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.335604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e066a9-e974-433a-8e1c-e70666e66308 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:43.337152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e066a9-e974-433a-8e1c-e70666e66308 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:43.338869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 179/354] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:43 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:43.340049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea27bd-24d0-4e60-aa59-a5946c6d90a1 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.340327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea27bd-24d0-4e60-aa59-a5946c6d90a1 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.340605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea27bd-24d0-4e60-aa59-a5946c6d90a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.366410 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-62ea27bd-24d0-4e60-aa59-a5946c6d90a1 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:43.367588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-62ea27bd-24d0-4e60-aa59-a5946c6d90a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:43.404359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:43.459815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55694e17-05aa-456a-ad0b-61a6c6281cec None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:43.460598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 180/355] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:43.461444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea27bd-24d0-4e60-aa59-a5946c6d90a1 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:43.507852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a25f8d-d948-4bf7-b1b2-6350ad1a7c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:43.507852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a25f8d-d948-4bf7-b1b2-6350ad1a7c30 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.508146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a25f8d-d948-4bf7-b1b2-6350ad1a7c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.508406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a25f8d-d948-4bf7-b1b2-6350ad1a7c30 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.509540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a25f8d-d948-4bf7-b1b2-6350ad1a7c30 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:43.511167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a25f8d-d948-4bf7-b1b2-6350ad1a7c30 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:43.512927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 176/356] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:43 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:43.513539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a25f8d-d948-4bf7-b1b2-6350ad1a7c30 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:43.515197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:43.607708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea27bd-24d0-4e60-aa59-a5946c6d90a1 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:43.639887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a25f8d-d948-4bf7-b1b2-6350ad1a7c30 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:43.645914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fef0f7-a142-48ef-8683-e1374a76fa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:43.646526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fef0f7-a142-48ef-8683-e1374a76fa88 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.646861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fef0f7-a142-48ef-8683-e1374a76fa88 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.647130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fef0f7-a142-48ef-8683-e1374a76fa88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.647913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fef0f7-a142-48ef-8683-e1374a76fa88 None admin] RBAC: Adding query filter params (role.id=853e41e951d14277bb9e21a8861bdbf9, user.id=2ce6ac3f7afa4246b7d7258ed3a4b0c7, scope.system=all) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:43.648220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fef0f7-a142-48ef-8683-e1374a76fa88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:43.650276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fef0f7-a142-48ef-8683-e1374a76fa88 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:43.666624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 181/357] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Oct 5 20:08:43 2022] GET /identity/v3/role_assignments?user.id=2ce6ac3f7afa4246b7d7258ed3a4b0c7&scope.system=all&role.id=853e41e951d14277bb9e21a8861bdbf9 => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:43.667635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fef0f7-a142-48ef-8683-e1374a76fa88 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:43.679585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:43.701108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72793563-2368-4eff-b860-9f8a0bfc59a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:43.701786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72793563-2368-4eff-b860-9f8a0bfc59a8 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.702067 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72793563-2368-4eff-b860-9f8a0bfc59a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.702342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72793563-2368-4eff-b860-9f8a0bfc59a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.703138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72793563-2368-4eff-b860-9f8a0bfc59a8 None admin] RBAC: Adding query filter params (group.id=77ed0c80333e48488ea8194944fd70b1, role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=d78c4808482c4e74b6e8a3037b83de37) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:43.703446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72793563-2368-4eff-b860-9f8a0bfc59a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:43.704966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72793563-2368-4eff-b860-9f8a0bfc59a8 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:43.724364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 177/358] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:43 2022] GET /identity/v3/role_assignments?group.id=77ed0c80333e48488ea8194944fd70b1&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=84d6c85b41034cae925afeb8921381e8 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:43.725232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72793563-2368-4eff-b860-9f8a0bfc59a8 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:43.881651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:43.884932 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fef0f7-a142-48ef-8683-e1374a76fa88 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:43.912320 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72793563-2368-4eff-b860-9f8a0bfc59a8 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:43.926522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ee126-1321-4376-9d02-bfdad44a2b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:43.927173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ee126-1321-4376-9d02-bfdad44a2b41 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.927448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ee126-1321-4376-9d02-bfdad44a2b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.927717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ee126-1321-4376-9d02-bfdad44a2b41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.941901 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ee126-1321-4376-9d02-bfdad44a2b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-d74ee126-1321-4376-9d02-bfdad44a2b41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:43.949299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:43.950390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:43.951206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 182/359] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:43.951879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ee126-1321-4376-9d02-bfdad44a2b41 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:43.962856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3c4fc-84d9-404a-a84a-aed214dfde03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:43.963855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3c4fc-84d9-404a-a84a-aed214dfde03 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.965086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3c4fc-84d9-404a-a84a-aed214dfde03 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.965086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3c4fc-84d9-404a-a84a-aed214dfde03 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.966696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3c4fc-84d9-404a-a84a-aed214dfde03 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:43.968388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b3c4fc-84d9-404a-a84a-aed214dfde03 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:43.970117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 178/360] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:43 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:43.971608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc7e28-065b-4c2f-9b03-681760491880 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:43.972436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc7e28-065b-4c2f-9b03-681760491880 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:43.972716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc7e28-065b-4c2f-9b03-681760491880 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:43.974332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 179/361] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:43.975120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc7e28-065b-4c2f-9b03-681760491880 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:44.056218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:44.074510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ee126-1321-4376-9d02-bfdad44a2b41 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:44.113218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc7e28-065b-4c2f-9b03-681760491880 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:44.125054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073a880-edeb-4a46-bbf3-17db9a04a37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:44.125652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073a880-edeb-4a46-bbf3-17db9a04a37c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:44.125866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073a880-edeb-4a46-bbf3-17db9a04a37c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:44.126133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073a880-edeb-4a46-bbf3-17db9a04a37c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:44.140203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073a880-edeb-4a46-bbf3-17db9a04a37c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-f073a880-edeb-4a46-bbf3-17db9a04a37c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:44.147641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:44.148518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:44.149299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:44.150950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 183/362] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:44.151595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073a880-edeb-4a46-bbf3-17db9a04a37c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:44.163606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69a46c-9dee-4a48-8ccb-6989426190a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:44.165834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69a46c-9dee-4a48-8ccb-6989426190a0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:44.165834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69a46c-9dee-4a48-8ccb-6989426190a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:44.165834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69a46c-9dee-4a48-8ccb-6989426190a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:44.165834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69a46c-9dee-4a48-8ccb-6989426190a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:44.166149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69a46c-9dee-4a48-8ccb-6989426190a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:44.167466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69a46c-9dee-4a48-8ccb-6989426190a0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:44.171981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:44.180403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 180/363] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:44.180403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4567a-0cdb-4065-817d-beb97b4981ea None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:44.180403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4567a-0cdb-4065-817d-beb97b4981ea None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:44.180703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4567a-0cdb-4065-817d-beb97b4981ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:44.202490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7bb4567a-0cdb-4065-817d-beb97b4981ea None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:44.203857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7bb4567a-0cdb-4065-817d-beb97b4981ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:44.282453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073a880-edeb-4a46-bbf3-17db9a04a37c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:44.290821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 181/364] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:44.292379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4567a-0cdb-4065-817d-beb97b4981ea None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:44.338705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78590f-6fcc-46b6-9f75-1cccc2553a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:44.339583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78590f-6fcc-46b6-9f75-1cccc2553a64 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:44.340137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78590f-6fcc-46b6-9f75-1cccc2553a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:44.340529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78590f-6fcc-46b6-9f75-1cccc2553a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:44.341402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78590f-6fcc-46b6-9f75-1cccc2553a64 None admin] RBAC: Adding query filter params (group.id=77ed0c80333e48488ea8194944fd70b1, role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=d78c4808482c4e74b6e8a3037b83de37) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:44.341905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78590f-6fcc-46b6-9f75-1cccc2553a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:44.343773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78590f-6fcc-46b6-9f75-1cccc2553a64 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:44.351893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 184/365] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:44 2022] GET /identity/v3/role_assignments?group.id=77ed0c80333e48488ea8194944fd70b1&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:44.353016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78590f-6fcc-46b6-9f75-1cccc2553a64 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:44.354849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:44.420691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4567a-0cdb-4065-817d-beb97b4981ea None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:44.454873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:44.467375 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f246f1-d5f2-4a93-9b0c-17ba366d1a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:44.468039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f246f1-d5f2-4a93-9b0c-17ba366d1a3c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:44.468321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f246f1-d5f2-4a93-9b0c-17ba366d1a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:44.468621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f246f1-d5f2-4a93-9b0c-17ba366d1a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:44.469615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f246f1-d5f2-4a93-9b0c-17ba366d1a3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:44.469925 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f246f1-d5f2-4a93-9b0c-17ba366d1a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:44.472572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f246f1-d5f2-4a93-9b0c-17ba366d1a3c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:44.483203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 182/366] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:44.485534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78590f-6fcc-46b6-9f75-1cccc2553a64 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:44.490620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f246f1-d5f2-4a93-9b0c-17ba366d1a3c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:44.553849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40001225-10d0-4283-9020-4d388139d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:44.554477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40001225-10d0-4283-9020-4d388139d295 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:44.554751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40001225-10d0-4283-9020-4d388139d295 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:44.555062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40001225-10d0-4283-9020-4d388139d295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:44.629458 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f246f1-d5f2-4a93-9b0c-17ba366d1a3c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:44.649815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:44.679391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a4289-502c-46f2-9285-6480600214a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:44.680271 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a4289-502c-46f2-9285-6480600214a2 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:44.680717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a4289-502c-46f2-9285-6480600214a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:44.681774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a4289-502c-46f2-9285-6480600214a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:44.684137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40001225-10d0-4283-9020-4d388139d295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:44.685702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40001225-10d0-4283-9020-4d388139d295 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:44.782303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a4289-502c-46f2-9285-6480600214a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:44.784896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a4289-502c-46f2-9285-6480600214a2 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:44.787140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:44.816740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 185/367] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:44.817772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40001225-10d0-4283-9020-4d388139d295 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:44.931383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40001225-10d0-4283-9020-4d388139d295 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:44.933247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 183/368] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:44.934206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a4289-502c-46f2-9285-6480600214a2 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:44.952736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:44.990082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff8246-e802-4cd5-94e7-3432bc39eb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:44.990714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff8246-e802-4cd5-94e7-3432bc39eb31 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:44.990987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff8246-e802-4cd5-94e7-3432bc39eb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:44.991252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff8246-e802-4cd5-94e7-3432bc39eb31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.005569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff8246-e802-4cd5-94e7-3432bc39eb31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-89ff8246-e802-4cd5-94e7-3432bc39eb31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:45.010362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:45.011221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:45.011993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 186/369] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:08:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:45.012633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff8246-e802-4cd5-94e7-3432bc39eb31 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:45.058043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:45.064674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a4289-502c-46f2-9285-6480600214a2 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:45.104946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd797a-39e8-4a64-9302-b94f3f77eab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:45.106073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd797a-39e8-4a64-9302-b94f3f77eab4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:45.106643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd797a-39e8-4a64-9302-b94f3f77eab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:45.107126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd797a-39e8-4a64-9302-b94f3f77eab4 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.108475 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd797a-39e8-4a64-9302-b94f3f77eab4 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:45.110326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd797a-39e8-4a64-9302-b94f3f77eab4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:45.112389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 184/370] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:44 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:45.113412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd797a-39e8-4a64-9302-b94f3f77eab4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:45.121354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff8246-e802-4cd5-94e7-3432bc39eb31 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:45.156293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc52ff-2ae0-4436-8b71-769fb587aeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:45.157018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc52ff-2ae0-4436-8b71-769fb587aeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:45.157198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc52ff-2ae0-4436-8b71-769fb587aeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:45.157470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc52ff-2ae0-4436-8b71-769fb587aeb5 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.159264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc52ff-2ae0-4436-8b71-769fb587aeb5 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:45.160708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc52ff-2ae0-4436-8b71-769fb587aeb5 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:45.163716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 187/371] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:45 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:45.164330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc52ff-2ae0-4436-8b71-769fb587aeb5 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:45.217369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:45.238212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd797a-39e8-4a64-9302-b94f3f77eab4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:45.270970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc52ff-2ae0-4436-8b71-769fb587aeb5 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:45.277930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf119ed-818a-447d-b479-1ef2fc8e9e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:45.278713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:45.279989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf119ed-818a-447d-b479-1ef2fc8e9e28 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:45.280485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf119ed-818a-447d-b479-1ef2fc8e9e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:45.280969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf119ed-818a-447d-b479-1ef2fc8e9e28 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.282442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf119ed-818a-447d-b479-1ef2fc8e9e28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:45.282944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf119ed-818a-447d-b479-1ef2fc8e9e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:45.285284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf119ed-818a-447d-b479-1ef2fc8e9e28 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:45.295550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 185/372] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:45.297626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf119ed-818a-447d-b479-1ef2fc8e9e28 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:45.313569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ead97-1eb9-4b84-8a53-2ee4773a4f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:45.314505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ead97-1eb9-4b84-8a53-2ee4773a4f80 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:45.314791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ead97-1eb9-4b84-8a53-2ee4773a4f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:45.315062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ead97-1eb9-4b84-8a53-2ee4773a4f80 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.316676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ead97-1eb9-4b84-8a53-2ee4773a4f80 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:45.318161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ead97-1eb9-4b84-8a53-2ee4773a4f80 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:45.320017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 188/373] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:45 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:45.320783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ead97-1eb9-4b84-8a53-2ee4773a4f80 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:45.422224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf119ed-818a-447d-b479-1ef2fc8e9e28 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:45.440062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ead97-1eb9-4b84-8a53-2ee4773a4f80 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:45.466024 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da8a0b-a675-4378-bf93-922bd2601280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:45.467133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da8a0b-a675-4378-bf93-922bd2601280 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:45.467675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da8a0b-a675-4378-bf93-922bd2601280 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:45.468188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da8a0b-a675-4378-bf93-922bd2601280 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.470493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da8a0b-a675-4378-bf93-922bd2601280 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:45.472440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da8a0b-a675-4378-bf93-922bd2601280 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:45.474269 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 186/374] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:45 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:45.475306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da8a0b-a675-4378-bf93-922bd2601280 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:45.479868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5ab40-cc00-4f8d-8d0b-3d906b012432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:45.480184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5ab40-cc00-4f8d-8d0b-3d906b012432 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:45.480384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5ab40-cc00-4f8d-8d0b-3d906b012432 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:45.480658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5ab40-cc00-4f8d-8d0b-3d906b012432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.505924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:45.535056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:45.566884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5ab40-cc00-4f8d-8d0b-3d906b012432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:45.568517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5ab40-cc00-4f8d-8d0b-3d906b012432 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:45.615704 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da8a0b-a675-4378-bf93-922bd2601280 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:45.655010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcaa83-dda3-427c-bc9a-891eade9405a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:45.655622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcaa83-dda3-427c-bc9a-891eade9405a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:45.655901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcaa83-dda3-427c-bc9a-891eade9405a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:45.656581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcaa83-dda3-427c-bc9a-891eade9405a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.656894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcaa83-dda3-427c-bc9a-891eade9405a None admin] RBAC: Adding query filter params (group.id=77ed0c80333e48488ea8194944fd70b1, role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:45.657201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcaa83-dda3-427c-bc9a-891eade9405a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:45.659249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcaa83-dda3-427c-bc9a-891eade9405a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:45.665264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 187/375] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:45 2022] GET /identity/v3/role_assignments?group.id=77ed0c80333e48488ea8194944fd70b1&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=84d6c85b41034cae925afeb8921381e8 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:45.666103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcaa83-dda3-427c-bc9a-891eade9405a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:45.705315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 189/376] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:45.710661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5ab40-cc00-4f8d-8d0b-3d906b012432 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:45.765542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:45.797451 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcaa83-dda3-427c-bc9a-891eade9405a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:45.818128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:45.839391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4f68e-e8bc-4a08-b1c0-e58044a904cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:45.839991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4f68e-e8bc-4a08-b1c0-e58044a904cb None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:45.840269 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4f68e-e8bc-4a08-b1c0-e58044a904cb None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:45.840537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4f68e-e8bc-4a08-b1c0-e58044a904cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.841274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4f68e-e8bc-4a08-b1c0-e58044a904cb None admin] RBAC: Adding query filter params (group.id=77ed0c80333e48488ea8194944fd70b1, role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:45.841643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4f68e-e8bc-4a08-b1c0-e58044a904cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:45.843412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4f68e-e8bc-4a08-b1c0-e58044a904cb None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:45.852097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 188/377] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:45 2022] GET /identity/v3/role_assignments?group.id=77ed0c80333e48488ea8194944fd70b1&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:45.854255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5ab40-cc00-4f8d-8d0b-3d906b012432 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:45.893085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e304af0-3614-484a-9fa4-3722ec4037f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:45.893976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e304af0-3614-484a-9fa4-3722ec4037f1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:45.894413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e304af0-3614-484a-9fa4-3722ec4037f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:45.894806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e304af0-3614-484a-9fa4-3722ec4037f1 None admin] PATH_INFO: `/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:45.896264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e304af0-3614-484a-9fa4-3722ec4037f1 None admin] RBAC: Authorizing `identity:get_project(project_id=e6d76b1c203e49b8ae5f7f394a20a597)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:45.898111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e304af0-3614-484a-9fa4-3722ec4037f1 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:45.900352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 190/378] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:45 2022] GET /identity/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:45.905801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4f68e-e8bc-4a08-b1c0-e58044a904cb None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:46.055667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4f68e-e8bc-4a08-b1c0-e58044a904cb None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:46.098396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b0293-67dc-4678-8554-918b40e93a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:46.099060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b0293-67dc-4678-8554-918b40e93a05 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:46.099337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b0293-67dc-4678-8554-918b40e93a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:46.099609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b0293-67dc-4678-8554-918b40e93a05 None admin] PATH_INFO: `/v3/groups/8eea4665d08047158fbd6ca8077179cb` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:46.106771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b0293-67dc-4678-8554-918b40e93a05 None admin] RBAC: Authorizing `identity:get_group(group_id=8eea4665d08047158fbd6ca8077179cb)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:46.108347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b0293-67dc-4678-8554-918b40e93a05 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:46.110433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 189/379] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:45 2022] GET /identity/v3/groups/8eea4665d08047158fbd6ca8077179cb => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:46.116967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e304af0-3614-484a-9fa4-3722ec4037f1 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:46.241130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e304af0-3614-484a-9fa4-3722ec4037f1 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:46.274030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b6948-f110-4c10-9746-cd11d7273158 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:46.274360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b6948-f110-4c10-9746-cd11d7273158 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:46.274570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b6948-f110-4c10-9746-cd11d7273158 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:46.278580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 190/380] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:46.283501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1eef0-6358-435f-978f-68b23441aeaf None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:46.283781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1eef0-6358-435f-978f-68b23441aeaf None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:46.284054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1eef0-6358-435f-978f-68b23441aeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:46.284852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d212ca-f59e-4a01-aba3-75d2b4e7feaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:46.285465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d212ca-f59e-4a01-aba3-75d2b4e7feaf None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:46.285775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d212ca-f59e-4a01-aba3-75d2b4e7feaf None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:46.286047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d212ca-f59e-4a01-aba3-75d2b4e7feaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:46.286874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d212ca-f59e-4a01-aba3-75d2b4e7feaf None admin] RBAC: Adding query filter params (group.id=8eea4665d08047158fbd6ca8077179cb, role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=e6d76b1c203e49b8ae5f7f394a20a597) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:46.287189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d212ca-f59e-4a01-aba3-75d2b4e7feaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:46.288619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d212ca-f59e-4a01-aba3-75d2b4e7feaf None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:46.294620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 191/381] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:46 2022] GET /identity/v3/role_assignments?group.id=8eea4665d08047158fbd6ca8077179cb&scope.project.id=e6d76b1c203e49b8ae5f7f394a20a597&role.id=36331ec562ff42f883fc883363bc4c8d => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:46.308783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:46.326750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-bea1eef0-6358-435f-978f-68b23441aeaf None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:46.327984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-bea1eef0-6358-435f-978f-68b23441aeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:46.425568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 191/382] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:46.749339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d212ca-f59e-4a01-aba3-75d2b4e7feaf None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:46.760144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:46.887393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d212ca-f59e-4a01-aba3-75d2b4e7feaf None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:46.932192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e27bea-c388-4ff0-8286-e0a3bd0e0f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:46.932830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e27bea-c388-4ff0-8286-e0a3bd0e0f60 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:46.933111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e27bea-c388-4ff0-8286-e0a3bd0e0f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:46.933381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e27bea-c388-4ff0-8286-e0a3bd0e0f60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:46.950122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e27bea-c388-4ff0-8286-e0a3bd0e0f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-35e27bea-c388-4ff0-8286-e0a3bd0e0f60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:46.955534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:46.956370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:46.957154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 192/383] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:46.962718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1eef0-6358-435f-978f-68b23441aeaf None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:47.100237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1eef0-6358-435f-978f-68b23441aeaf None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:47.144525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8085f-9fbb-4242-9785-8d7ea48d1367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:47.145146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8085f-9fbb-4242-9785-8d7ea48d1367 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:47.145423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8085f-9fbb-4242-9785-8d7ea48d1367 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:47.145715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8085f-9fbb-4242-9785-8d7ea48d1367 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:47.146376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8085f-9fbb-4242-9785-8d7ea48d1367 None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:47.146674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8085f-9fbb-4242-9785-8d7ea48d1367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:47.148237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8085f-9fbb-4242-9785-8d7ea48d1367 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:47.149805 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:47.157684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 192/384] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:47.164674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e27bea-c388-4ff0-8286-e0a3bd0e0f60 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:47.250305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d23d-df76-4d80-bd3e-c50a3ec48ea6 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:47.250692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d23d-df76-4d80-bd3e-c50a3ec48ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:47.250868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d23d-df76-4d80-bd3e-c50a3ec48ea6 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:47.253267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 193/385] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:08:47.260103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568563e-e0b9-4d5f-ad56-bb1aa2917d5b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:47.260390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568563e-e0b9-4d5f-ad56-bb1aa2917d5b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:47.260682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568563e-e0b9-4d5f-ad56-bb1aa2917d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:47.264313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:47.288832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9568563e-e0b9-4d5f-ad56-bb1aa2917d5b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:47.294389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9568563e-e0b9-4d5f-ad56-bb1aa2917d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:47.306543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e27bea-c388-4ff0-8286-e0a3bd0e0f60 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:47.369428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a72ee-a448-4cb9-b33f-1125a555cf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:47.370015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a72ee-a448-4cb9-b33f-1125a555cf97 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:47.370332 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a72ee-a448-4cb9-b33f-1125a555cf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:47.370605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a72ee-a448-4cb9-b33f-1125a555cf97 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:47.372329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a72ee-a448-4cb9-b33f-1125a555cf97 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:47.373820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a72ee-a448-4cb9-b33f-1125a555cf97 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:47.375387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 193/386] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:47 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:47.382926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a72ee-a448-4cb9-b33f-1125a555cf97 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:47.425028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 194/387] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:47.539658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a72ee-a448-4cb9-b33f-1125a555cf97 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:47.578564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323cf2e-2edc-4ea1-9bba-bbf248e4659c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:47.579200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323cf2e-2edc-4ea1-9bba-bbf248e4659c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:47.579480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323cf2e-2edc-4ea1-9bba-bbf248e4659c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:47.579751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323cf2e-2edc-4ea1-9bba-bbf248e4659c None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:47.581570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323cf2e-2edc-4ea1-9bba-bbf248e4659c None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:47.583015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323cf2e-2edc-4ea1-9bba-bbf248e4659c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:47.584988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 194/388] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:47 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:47.590745 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568563e-e0b9-4d5f-ad56-bb1aa2917d5b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:47.709904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568563e-e0b9-4d5f-ad56-bb1aa2917d5b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:47.747272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:47.759291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb099c-77aa-481d-b92b-460bf3273741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:47.759954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb099c-77aa-481d-b92b-460bf3273741 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:47.760237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb099c-77aa-481d-b92b-460bf3273741 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:47.760509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb099c-77aa-481d-b92b-460bf3273741 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37/groups/77ed0c80333e48488ea8194944fd70b1/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:47.766735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb099c-77aa-481d-b92b-460bf3273741 None admin] RBAC: Authorizing `identity:create_grant(project_id=d78c4808482c4e74b6e8a3037b83de37, group_id=77ed0c80333e48488ea8194944fd70b1, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:47.768767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb099c-77aa-481d-b92b-460bf3273741 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:47.796556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 195/389] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 20:08:47 2022] PUT /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37/groups/77ed0c80333e48488ea8194944fd70b1/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:48.065382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323cf2e-2edc-4ea1-9bba-bbf248e4659c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:48.068358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:48.190707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323cf2e-2edc-4ea1-9bba-bbf248e4659c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:48.226897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98daf9-f914-4812-b220-7a0077409572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:48.227531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98daf9-f914-4812-b220-7a0077409572 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:48.227808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98daf9-f914-4812-b220-7a0077409572 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:48.228081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98daf9-f914-4812-b220-7a0077409572 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:48.239335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98daf9-f914-4812-b220-7a0077409572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-7c98daf9-f914-4812-b220-7a0077409572 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:48.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:48.245601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:48.246459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 195/390] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:48.252779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb099c-77aa-481d-b92b-460bf3273741 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:48.370366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb099c-77aa-481d-b92b-460bf3273741 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:48.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e02979-dd9c-4286-879a-bcba01a3b4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:48.413581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02979-dd9c-4286-879a-bcba01a3b4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:48.413862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02979-dd9c-4286-879a-bcba01a3b4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:48.414130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e02979-dd9c-4286-879a-bcba01a3b4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:48.414798 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02979-dd9c-4286-879a-bcba01a3b4b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:48.415154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02979-dd9c-4286-879a-bcba01a3b4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:48.416521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e02979-dd9c-4286-879a-bcba01a3b4b5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:48.426553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 196/391] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:48.432159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98daf9-f914-4812-b220-7a0077409572 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:48.499446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:48.500830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f17c4-5102-4fa3-89db-a701ddd50515 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:48.501117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f17c4-5102-4fa3-89db-a701ddd50515 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:48.501446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f17c4-5102-4fa3-89db-a701ddd50515 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:48.503861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 197/392] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:08:48.510944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78214a6f-3009-4262-be97-b72c536367be None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:48.511304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78214a6f-3009-4262-be97-b72c536367be None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:48.511781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78214a6f-3009-4262-be97-b72c536367be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:48.542700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-78214a6f-3009-4262-be97-b72c536367be None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:48.545342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-78214a6f-3009-4262-be97-b72c536367be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:48.564906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98daf9-f914-4812-b220-7a0077409572 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:48.637494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07847b4-5f74-4bb4-9647-4c5a5df5d141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:48.638084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07847b4-5f74-4bb4-9647-4c5a5df5d141 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:48.638342 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07847b4-5f74-4bb4-9647-4c5a5df5d141 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:48.638623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07847b4-5f74-4bb4-9647-4c5a5df5d141 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:48.643491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07847b4-5f74-4bb4-9647-4c5a5df5d141 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:48.645933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07847b4-5f74-4bb4-9647-4c5a5df5d141 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:48.646455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:48.648104 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 196/393] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:48 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:48.654336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07847b4-5f74-4bb4-9647-4c5a5df5d141 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:48.686326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 198/394] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:48.763081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ff505-f9e2-4d1b-bf5e-38c7d23bdc36 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:48.763404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ff505-f9e2-4d1b-bf5e-38c7d23bdc36 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:48.763617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ff505-f9e2-4d1b-bf5e-38c7d23bdc36 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:48.766783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 199/395] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:08:48.771516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c01b-e900-4608-97f7-68ed3da4f211 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:48.771816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c01b-e900-4608-97f7-68ed3da4f211 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:48.772085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c01b-e900-4608-97f7-68ed3da4f211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:48.807011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07847b4-5f74-4bb4-9647-4c5a5df5d141 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:48.814819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-6c50c01b-e900-4608-97f7-68ed3da4f211 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:48.816014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6c50c01b-e900-4608-97f7-68ed3da4f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:48.843618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:48.848997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16b6f-ac3d-4684-a05f-e8cbc8a840ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:48.850001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16b6f-ac3d-4684-a05f-e8cbc8a840ce None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:48.850543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16b6f-ac3d-4684-a05f-e8cbc8a840ce None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:48.850963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16b6f-ac3d-4684-a05f-e8cbc8a840ce None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:48.852398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b16b6f-ac3d-4684-a05f-e8cbc8a840ce None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:48.854614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b16b6f-ac3d-4684-a05f-e8cbc8a840ce None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:48.867333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 197/396] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:48 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:48.869623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16b6f-ac3d-4684-a05f-e8cbc8a840ce None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:48.964356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 200/397] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:48.989634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16b6f-ac3d-4684-a05f-e8cbc8a840ce None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:49.027021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f798b-8089-46e8-953b-df27e0202eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:49.027988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f798b-8089-46e8-953b-df27e0202eac None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:49.028435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f798b-8089-46e8-953b-df27e0202eac None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:49.028854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f798b-8089-46e8-953b-df27e0202eac None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37/groups/77ed0c80333e48488ea8194944fd70b1/roles/6a600852dc8442fdb2ffc3d663e274ff` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:49.031382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f798b-8089-46e8-953b-df27e0202eac None admin] RBAC: Authorizing `identity:create_grant(project_id=d78c4808482c4e74b6e8a3037b83de37, group_id=77ed0c80333e48488ea8194944fd70b1, role_id=6a600852dc8442fdb2ffc3d663e274ff)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:49.033004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f798b-8089-46e8-953b-df27e0202eac None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:49.047899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 198/398] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 20:08:48 2022] PUT /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37/groups/77ed0c80333e48488ea8194944fd70b1/roles/6a600852dc8442fdb2ffc3d663e274ff => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:49.101818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c01b-e900-4608-97f7-68ed3da4f211 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:49.222820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c01b-e900-4608-97f7-68ed3da4f211 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:49.233215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:49.263603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8cf36-c9a7-4272-ba6c-db2b681a540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:49.264592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8cf36-c9a7-4272-ba6c-db2b681a540d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:49.265072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8cf36-c9a7-4272-ba6c-db2b681a540d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:49.265707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8cf36-c9a7-4272-ba6c-db2b681a540d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:49.277366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e8cf36-c9a7-4272-ba6c-db2b681a540d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-05e8cf36-c9a7-4272-ba6c-db2b681a540d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:49.283138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:49.284073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:49.284848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:49.287326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 201/399] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:49.292940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f798b-8089-46e8-953b-df27e0202eac None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:49.361927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:49.406485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8cf36-c9a7-4272-ba6c-db2b681a540d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:49.411655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f798b-8089-46e8-953b-df27e0202eac None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:49.448769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4e8a9-89cd-4765-a8e3-c8a0713cec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:49.449381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4e8a9-89cd-4765-a8e3-c8a0713cec43 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:49.449692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4e8a9-89cd-4765-a8e3-c8a0713cec43 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:49.449966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4e8a9-89cd-4765-a8e3-c8a0713cec43 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:49.450630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4e8a9-89cd-4765-a8e3-c8a0713cec43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:49.450930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4e8a9-89cd-4765-a8e3-c8a0713cec43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:49.452307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4e8a9-89cd-4765-a8e3-c8a0713cec43 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:49.462590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 199/400] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:49.474094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4e8a9-89cd-4765-a8e3-c8a0713cec43 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:49.575719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8cf36-c9a7-4272-ba6c-db2b681a540d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:49.594186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4e8a9-89cd-4765-a8e3-c8a0713cec43 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:49.618408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:49.619941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07be36-0770-42b9-a70a-c7589295897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:49.621056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07be36-0770-42b9-a70a-c7589295897d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:49.621750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07be36-0770-42b9-a70a-c7589295897d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:49.622163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07be36-0770-42b9-a70a-c7589295897d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:49.634366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07be36-0770-42b9-a70a-c7589295897d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:49.635889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfd9cc-a4e1-40ee-bf9b-cc7cb69b5852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:49.636577 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfd9cc-a4e1-40ee-bf9b-cc7cb69b5852 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:49.636854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfd9cc-a4e1-40ee-bf9b-cc7cb69b5852 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:49.637125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfd9cc-a4e1-40ee-bf9b-cc7cb69b5852 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:49.638382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfd9cc-a4e1-40ee-bf9b-cc7cb69b5852 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:49.639902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfd9cc-a4e1-40ee-bf9b-cc7cb69b5852 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-7a07be36-0770-42b9-a70a-c7589295897d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:49.640730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:49.641570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:49.642351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:49.643474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 200/401] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:49 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:49.643656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a73560-5c92-43e4-aad5-cc96d8e52206 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:49.644001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a73560-5c92-43e4-aad5-cc96d8e52206 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:49.644245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a73560-5c92-43e4-aad5-cc96d8e52206 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:49.649538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 202/403] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:49.649694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 201/403] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:49.650386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a73560-5c92-43e4-aad5-cc96d8e52206 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:49.651210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a491c7a-8108-463d-bca1-c2bd46ea3ff1 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:49.651633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a491c7a-8108-463d-bca1-c2bd46ea3ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:49.652039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a491c7a-8108-463d-bca1-c2bd46ea3ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:49.675320 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-5a491c7a-8108-463d-bca1-c2bd46ea3ff1 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:49.677319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5a491c7a-8108-463d-bca1-c2bd46ea3ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:49.708964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:49.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 203/404] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:49.766095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a491c7a-8108-463d-bca1-c2bd46ea3ff1 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:49.790010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a73560-5c92-43e4-aad5-cc96d8e52206 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:49.855113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0510c6-4b89-4397-851b-afe9472dba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:49.856097 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0510c6-4b89-4397-851b-afe9472dba0f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:49.856563 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0510c6-4b89-4397-851b-afe9472dba0f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:49.857077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0510c6-4b89-4397-851b-afe9472dba0f None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:49.859389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0510c6-4b89-4397-851b-afe9472dba0f None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:49.862006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0510c6-4b89-4397-851b-afe9472dba0f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:49.863963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 202/405] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:49 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:49.879608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0510c6-4b89-4397-851b-afe9472dba0f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:49.947042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a491c7a-8108-463d-bca1-c2bd46ea3ff1 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:49.975650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:50.014581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140808ff-03f9-4eeb-afa0-8087f35b9601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:50.015612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140808ff-03f9-4eeb-afa0-8087f35b9601 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.016080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140808ff-03f9-4eeb-afa0-8087f35b9601 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.016551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140808ff-03f9-4eeb-afa0-8087f35b9601 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.017412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140808ff-03f9-4eeb-afa0-8087f35b9601 None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:50.018532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140808ff-03f9-4eeb-afa0-8087f35b9601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:50.020140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140808ff-03f9-4eeb-afa0-8087f35b9601 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:50.038138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 204/406] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:50.043595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140808ff-03f9-4eeb-afa0-8087f35b9601 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:50.056283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0510c6-4b89-4397-851b-afe9472dba0f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:50.067106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:50.105781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8ec70-129b-4038-b2a7-0c0aeedef664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:50.106444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8ec70-129b-4038-b2a7-0c0aeedef664 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.106719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8ec70-129b-4038-b2a7-0c0aeedef664 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.107017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8ec70-129b-4038-b2a7-0c0aeedef664 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/groups/77ed0c80333e48488ea8194944fd70b1/roles/6a600852dc8442fdb2ffc3d663e274ff` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.110819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8ec70-129b-4038-b2a7-0c0aeedef664 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9189f4b1cc4920b3079cdc0d358919, group_id=77ed0c80333e48488ea8194944fd70b1, role_id=6a600852dc8442fdb2ffc3d663e274ff)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:50.112320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8ec70-129b-4038-b2a7-0c0aeedef664 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:50.134996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 203/407] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 20:08:49 2022] PUT /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/groups/77ed0c80333e48488ea8194944fd70b1/roles/6a600852dc8442fdb2ffc3d663e274ff => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:50.136063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8ec70-129b-4038-b2a7-0c0aeedef664 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:50.165530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140808ff-03f9-4eeb-afa0-8087f35b9601 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:50.205848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2b39a-5500-4b5e-8991-d33a8b075213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:50.206892 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2b39a-5500-4b5e-8991-d33a8b075213 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.207383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2b39a-5500-4b5e-8991-d33a8b075213 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.207899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2b39a-5500-4b5e-8991-d33a8b075213 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.209410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2b39a-5500-4b5e-8991-d33a8b075213 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:50.211223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2b39a-5500-4b5e-8991-d33a8b075213 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:50.213553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 205/408] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:50 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:50.219565 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2b39a-5500-4b5e-8991-d33a8b075213 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:50.317579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8ec70-129b-4038-b2a7-0c0aeedef664 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:50.353883 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:50.388610 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a702a-c8f8-4e02-81ab-24530b190e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:50.389228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a702a-c8f8-4e02-81ab-24530b190e99 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.389503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a702a-c8f8-4e02-81ab-24530b190e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.389788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a702a-c8f8-4e02-81ab-24530b190e99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.395805 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:50.408065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a702a-c8f8-4e02-81ab-24530b190e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:50.409558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2b39a-5500-4b5e-8991-d33a8b075213 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-023a702a-c8f8-4e02-81ab-24530b190e99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:50.415172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:50.416036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:50.416795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:50.419128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 204/409] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:08:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:50.425551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a702a-c8f8-4e02-81ab-24530b190e99 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:50.449706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eca536-fad4-48aa-a603-1da7363caea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:50.450339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eca536-fad4-48aa-a603-1da7363caea5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.450611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eca536-fad4-48aa-a603-1da7363caea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.450876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eca536-fad4-48aa-a603-1da7363caea5 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.452147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eca536-fad4-48aa-a603-1da7363caea5 None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:50.453629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eca536-fad4-48aa-a603-1da7363caea5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:50.456052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 206/410] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:50 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:50.461462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eca536-fad4-48aa-a603-1da7363caea5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:50.537846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a702a-c8f8-4e02-81ab-24530b190e99 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:50.573374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eca536-fad4-48aa-a603-1da7363caea5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:50.574528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779e104-416e-40ab-ba58-ae9093441e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:50.575324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779e104-416e-40ab-ba58-ae9093441e8a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.575749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779e104-416e-40ab-ba58-ae9093441e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.576188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779e104-416e-40ab-ba58-ae9093441e8a None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.577082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779e104-416e-40ab-ba58-ae9093441e8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:50.577606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779e104-416e-40ab-ba58-ae9093441e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:50.579394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779e104-416e-40ab-ba58-ae9093441e8a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:50.589330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 205/411] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:50.595475 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779e104-416e-40ab-ba58-ae9093441e8a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:50.612392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d35a3c-1588-4210-befb-0aaa6cf5f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:50.613057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d35a3c-1588-4210-befb-0aaa6cf5f5f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.613330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d35a3c-1588-4210-befb-0aaa6cf5f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.613686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d35a3c-1588-4210-befb-0aaa6cf5f5f5 None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/groups/77ed0c80333e48488ea8194944fd70b1/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.615699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d35a3c-1588-4210-befb-0aaa6cf5f5f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c9189f4b1cc4920b3079cdc0d358919, group_id=77ed0c80333e48488ea8194944fd70b1, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:50.617094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d35a3c-1588-4210-befb-0aaa6cf5f5f5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:50.634050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 207/412] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 20:08:50 2022] PUT /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919/groups/77ed0c80333e48488ea8194944fd70b1/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:08:50.645562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:50.714248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779e104-416e-40ab-ba58-ae9093441e8a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:50.754998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77770c6-62b8-4877-8af3-9e32e4ecf35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:50.756159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77770c6-62b8-4877-8af3-9e32e4ecf35e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.756668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77770c6-62b8-4877-8af3-9e32e4ecf35e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.757144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77770c6-62b8-4877-8af3-9e32e4ecf35e None admin] PATH_INFO: `/v3/groups/8eea4665d08047158fbd6ca8077179cb` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.758821 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77770c6-62b8-4877-8af3-9e32e4ecf35e None admin] RBAC: Authorizing `identity:get_group(group_id=8eea4665d08047158fbd6ca8077179cb)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:50.760537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77770c6-62b8-4877-8af3-9e32e4ecf35e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:50.762745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 206/413] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:50 2022] GET /identity/v3/groups/8eea4665d08047158fbd6ca8077179cb => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:50.768903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d35a3c-1588-4210-befb-0aaa6cf5f5f5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:50.802488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:50.923349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d35a3c-1588-4210-befb-0aaa6cf5f5f5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:50.954063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4afbd9-0336-4668-953e-e25041716234 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.954334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4afbd9-0336-4668-953e-e25041716234 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.954607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4afbd9-0336-4668-953e-e25041716234 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.959815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 207/414] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:08:50.961664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86d57a-7ed2-4ac8-ae90-320445414a80 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.961967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86d57a-7ed2-4ac8-ae90-320445414a80 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.962218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86d57a-7ed2-4ac8-ae90-320445414a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.966482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac666c-02d2-4407-a33b-ebfa9ac3c7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:50.967590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac666c-02d2-4407-a33b-ebfa9ac3c7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:50.968067 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac666c-02d2-4407-a33b-ebfa9ac3c7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:50.968662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac666c-02d2-4407-a33b-ebfa9ac3c7e6 None admin] PATH_INFO: `/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:50.970231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac666c-02d2-4407-a33b-ebfa9ac3c7e6 None admin] RBAC: Authorizing `identity:get_project(project_id=e6d76b1c203e49b8ae5f7f394a20a597)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:50.973307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac666c-02d2-4407-a33b-ebfa9ac3c7e6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:50.975586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 208/415] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:50 2022] GET /identity/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:50.982427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac666c-02d2-4407-a33b-ebfa9ac3c7e6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:50.987671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-be86d57a-7ed2-4ac8-ae90-320445414a80 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:50.988938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-be86d57a-7ed2-4ac8-ae90-320445414a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:51.074325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:51.092403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac666c-02d2-4407-a33b-ebfa9ac3c7e6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:51.131781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36247418-b73d-41bd-9f66-cba3da7e36f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:51.132773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36247418-b73d-41bd-9f66-cba3da7e36f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:51.133364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36247418-b73d-41bd-9f66-cba3da7e36f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:51.134338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36247418-b73d-41bd-9f66-cba3da7e36f4 None admin] PATH_INFO: `/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597/groups/8eea4665d08047158fbd6ca8077179cb/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:51.136585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36247418-b73d-41bd-9f66-cba3da7e36f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e6d76b1c203e49b8ae5f7f394a20a597, group_id=8eea4665d08047158fbd6ca8077179cb, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:51.138551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36247418-b73d-41bd-9f66-cba3da7e36f4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:51.154958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 209/416] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 20:08:50 2022] PUT /identity/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597/groups/8eea4665d08047158fbd6ca8077179cb/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:08:51.192777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 208/417] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:51.528847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df565ef-4ed6-4121-92f5-b800e4f18164 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:51.529409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df565ef-4ed6-4121-92f5-b800e4f18164 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:51.529987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df565ef-4ed6-4121-92f5-b800e4f18164 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:51.531940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 210/418] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:51.536984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c250063-9522-4ded-bdf8-1a7537c53686 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:51.537470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c250063-9522-4ded-bdf8-1a7537c53686 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:51.537943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c250063-9522-4ded-bdf8-1a7537c53686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:51.556634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6c250063-9522-4ded-bdf8-1a7537c53686 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:51.562155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6c250063-9522-4ded-bdf8-1a7537c53686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:51.568771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df565ef-4ed6-4121-92f5-b800e4f18164 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:51.618662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:51.687173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 209/419] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:51.719150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df565ef-4ed6-4121-92f5-b800e4f18164 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:51.769476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb684f-c65e-43e7-9ef2-27894e478c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:51.770136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb684f-c65e-43e7-9ef2-27894e478c01 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:51.770411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb684f-c65e-43e7-9ef2-27894e478c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:51.770674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb684f-c65e-43e7-9ef2-27894e478c01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:51.786793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb684f-c65e-43e7-9ef2-27894e478c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-65cb684f-c65e-43e7-9ef2-27894e478c01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:51.791500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:51.792344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:51.793111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 211/420] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:51.798571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c250063-9522-4ded-bdf8-1a7537c53686 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:51.940862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c250063-9522-4ded-bdf8-1a7537c53686 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:51.981216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbf265-1c60-4dc4-a441-6bd930f0a8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:51.982523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbf265-1c60-4dc4-a441-6bd930f0a8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:51.983004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbf265-1c60-4dc4-a441-6bd930f0a8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:51.983473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbf265-1c60-4dc4-a441-6bd930f0a8e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:51.984328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbf265-1c60-4dc4-a441-6bd930f0a8e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:51.984842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbf265-1c60-4dc4-a441-6bd930f0a8e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:51.986468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbf265-1c60-4dc4-a441-6bd930f0a8e6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:51.996040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 210/421] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:52.002580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb684f-c65e-43e7-9ef2-27894e478c01 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:52.009458 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:52.133186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbf265-1c60-4dc4-a441-6bd930f0a8e6 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:52.143210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb684f-c65e-43e7-9ef2-27894e478c01 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:52.181073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:52.190789 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b202f-4f9c-4604-8486-031e8fded012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:52.192128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b202f-4f9c-4604-8486-031e8fded012 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:52.192128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b202f-4f9c-4604-8486-031e8fded012 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:52.192128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b202f-4f9c-4604-8486-031e8fded012 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:52.312047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbf265-1c60-4dc4-a441-6bd930f0a8e6 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:52.328282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b202f-4f9c-4604-8486-031e8fded012 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:52.329749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b202f-4f9c-4604-8486-031e8fded012 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:52.332527 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:52.374740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994d06a-09af-45a7-ac9a-f9419306d4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:52.375769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994d06a-09af-45a7-ac9a-f9419306d4ba None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:52.376254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994d06a-09af-45a7-ac9a-f9419306d4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:52.376719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994d06a-09af-45a7-ac9a-f9419306d4ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:52.387631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994d06a-09af-45a7-ac9a-f9419306d4ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-c994d06a-09af-45a7-ac9a-f9419306d4ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:52.401022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:52.402042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:52.402799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:52.403531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 211/422] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:52.408972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994d06a-09af-45a7-ac9a-f9419306d4ba None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:52.455344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 212/423] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:52.457341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b202f-4f9c-4604-8486-031e8fded012 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:52.591942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:52.614842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994d06a-09af-45a7-ac9a-f9419306d4ba None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:52.635776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b202f-4f9c-4604-8486-031e8fded012 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:52.640930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:52.681366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5eead9-a093-488f-b431-8179aced9c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:52.683034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5eead9-a093-488f-b431-8179aced9c91 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:52.683495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5eead9-a093-488f-b431-8179aced9c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:52.684091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5eead9-a093-488f-b431-8179aced9c91 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:52.686125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5eead9-a093-488f-b431-8179aced9c91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:52.686682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5eead9-a093-488f-b431-8179aced9c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:52.688473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5eead9-a093-488f-b431-8179aced9c91 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:52.700878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961ec6-d5f6-450f-a273-5fe92cb1dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:52.701864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961ec6-d5f6-450f-a273-5fe92cb1dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:52.702376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961ec6-d5f6-450f-a273-5fe92cb1dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:52.702815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961ec6-d5f6-450f-a273-5fe92cb1dbc1 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:52.704248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7961ec6-d5f6-450f-a273-5fe92cb1dbc1 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:52.706477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7961ec6-d5f6-450f-a273-5fe92cb1dbc1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:52.708036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 212/424] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:52.712503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 213/425] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:52 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:52.713878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5eead9-a093-488f-b431-8179aced9c91 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:52.714868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961ec6-d5f6-450f-a273-5fe92cb1dbc1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:52.854713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961ec6-d5f6-450f-a273-5fe92cb1dbc1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:52.886470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5eead9-a093-488f-b431-8179aced9c91 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:52.899268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90f4ab-353d-4c63-9464-cb6efeefc1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:52.900152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90f4ab-353d-4c63-9464-cb6efeefc1dd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:52.900613 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90f4ab-353d-4c63-9464-cb6efeefc1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:52.901050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90f4ab-353d-4c63-9464-cb6efeefc1dd None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:52.902365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90f4ab-353d-4c63-9464-cb6efeefc1dd None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:52.904522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90f4ab-353d-4c63-9464-cb6efeefc1dd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:52.911871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 214/426] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:52 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:52.913752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a7e16-be75-425d-bfea-5a6b0cb85daa None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:52.914548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a7e16-be75-425d-bfea-5a6b0cb85daa None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:52.915262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a7e16-be75-425d-bfea-5a6b0cb85daa None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:52.916707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 215/427] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:52.921993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a7e16-be75-425d-bfea-5a6b0cb85daa None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:52.930638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa703e23-ebd1-4d46-aab4-4ccce8e9abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:52.931438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa703e23-ebd1-4d46-aab4-4ccce8e9abed None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:52.931849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa703e23-ebd1-4d46-aab4-4ccce8e9abed None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:52.932246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa703e23-ebd1-4d46-aab4-4ccce8e9abed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:52.952383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:52.974144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:53.007689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa703e23-ebd1-4d46-aab4-4ccce8e9abed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:53.009592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa703e23-ebd1-4d46-aab4-4ccce8e9abed None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:53.041394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a7e16-be75-425d-bfea-5a6b0cb85daa None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:53.079865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75875e6-d516-4567-a7b8-3e234510e989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:53.080683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75875e6-d516-4567-a7b8-3e234510e989 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.081131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75875e6-d516-4567-a7b8-3e234510e989 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.081567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75875e6-d516-4567-a7b8-3e234510e989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.082443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75875e6-d516-4567-a7b8-3e234510e989 None admin] RBAC: Adding query filter params (group.id=77ed0c80333e48488ea8194944fd70b1, role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=d78c4808482c4e74b6e8a3037b83de37) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:53.083051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75875e6-d516-4567-a7b8-3e234510e989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:53.085394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75875e6-d516-4567-a7b8-3e234510e989 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:53.101552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 216/428] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:52 2022] GET /identity/v3/role_assignments?group.id=77ed0c80333e48488ea8194944fd70b1&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=84d6c85b41034cae925afeb8921381e8 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:53.101552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce963680-f2b9-4205-93bd-44e6e8a3dd13 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.101552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce963680-f2b9-4205-93bd-44e6e8a3dd13 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.101552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce963680-f2b9-4205-93bd-44e6e8a3dd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.137798 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ce963680-f2b9-4205-93bd-44e6e8a3dd13 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:53.139009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ce963680-f2b9-4205-93bd-44e6e8a3dd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:53.144198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 213/429] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 20:08:53.151085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa703e23-ebd1-4d46-aab4-4ccce8e9abed None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:53.239248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:53.293939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 217/430] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:53.297888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faf7a2-fd27-4627-9e65-487027c1937e None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.298375 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faf7a2-fd27-4627-9e65-487027c1937e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.299601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37faf7a2-fd27-4627-9e65-487027c1937e None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.302905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 218/431] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:53.308603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c58ea8-ec63-45a5-a6b7-2876e123a70b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.308886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c58ea8-ec63-45a5-a6b7-2876e123a70b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.309155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c58ea8-ec63-45a5-a6b7-2876e123a70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.339583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a5c58ea8-ec63-45a5-a6b7-2876e123a70b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:53.341201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a5c58ea8-ec63-45a5-a6b7-2876e123a70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:53.347269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa703e23-ebd1-4d46-aab4-4ccce8e9abed None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:53.373827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:53.394923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc8e70-ce3a-4f35-9f78-1877e9e95161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:53.396430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc8e70-ce3a-4f35-9f78-1877e9e95161 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.396911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc8e70-ce3a-4f35-9f78-1877e9e95161 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.397377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc8e70-ce3a-4f35-9f78-1877e9e95161 None admin] PATH_INFO: `/v3/projects/d78c4808482c4e74b6e8a3037b83de37` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.399249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc8e70-ce3a-4f35-9f78-1877e9e95161 None admin] RBAC: Authorizing `identity:get_project(project_id=d78c4808482c4e74b6e8a3037b83de37)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:53.401770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc8e70-ce3a-4f35-9f78-1877e9e95161 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:53.404636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 214/432] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:53 2022] GET /identity/v3/projects/d78c4808482c4e74b6e8a3037b83de37 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:53.410803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc8e70-ce3a-4f35-9f78-1877e9e95161 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:53.428211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 219/433] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:53.528001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc8e70-ce3a-4f35-9f78-1877e9e95161 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:53.546701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:53.568330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723589f3-7b41-4163-8979-10e47d9b7e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:53.569331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723589f3-7b41-4163-8979-10e47d9b7e19 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.569838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723589f3-7b41-4163-8979-10e47d9b7e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.570326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723589f3-7b41-4163-8979-10e47d9b7e19 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.571667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723589f3-7b41-4163-8979-10e47d9b7e19 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:53.573413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723589f3-7b41-4163-8979-10e47d9b7e19 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:53.575568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 215/434] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:53 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:53.582288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c58ea8-ec63-45a5-a6b7-2876e123a70b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:53.598007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723589f3-7b41-4163-8979-10e47d9b7e19 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:53.704726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c58ea8-ec63-45a5-a6b7-2876e123a70b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:53.735270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723589f3-7b41-4163-8979-10e47d9b7e19 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:53.745274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710f6bd-55e7-45c4-8471-5c9acaadfcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:53.746382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710f6bd-55e7-45c4-8471-5c9acaadfcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.746852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710f6bd-55e7-45c4-8471-5c9acaadfcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.747387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710f6bd-55e7-45c4-8471-5c9acaadfcd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.748338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710f6bd-55e7-45c4-8471-5c9acaadfcd0 None admin] RBAC: Adding query filter params (group.id=77ed0c80333e48488ea8194944fd70b1, role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=d78c4808482c4e74b6e8a3037b83de37) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:53.748847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710f6bd-55e7-45c4-8471-5c9acaadfcd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:53.750651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710f6bd-55e7-45c4-8471-5c9acaadfcd0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:53.762189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 220/435] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:53 2022] GET /identity/v3/role_assignments?group.id=77ed0c80333e48488ea8194944fd70b1&scope.project.id=d78c4808482c4e74b6e8a3037b83de37&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:53.763451 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710f6bd-55e7-45c4-8471-5c9acaadfcd0 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:53.779662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43734c14-c2f6-474d-9908-44ad1fc6e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:53.780556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43734c14-c2f6-474d-9908-44ad1fc6e21d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.780973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43734c14-c2f6-474d-9908-44ad1fc6e21d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.781382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43734c14-c2f6-474d-9908-44ad1fc6e21d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.794605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43734c14-c2f6-474d-9908-44ad1fc6e21d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:53.799963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-43734c14-c2f6-474d-9908-44ad1fc6e21d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:53.802569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:53.803517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:08:53.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:53.806998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 216/436] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:08:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:53.813075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43734c14-c2f6-474d-9908-44ad1fc6e21d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:53.873382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:53.907210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710f6bd-55e7-45c4-8471-5c9acaadfcd0 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:53.944986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43734c14-c2f6-474d-9908-44ad1fc6e21d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:53.962114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d655f77-05c9-4347-88a5-639d5e7666a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:53.962650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d655f77-05c9-4347-88a5-639d5e7666a0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.963080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d655f77-05c9-4347-88a5-639d5e7666a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.963472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d655f77-05c9-4347-88a5-639d5e7666a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.977167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d655f77-05c9-4347-88a5-639d5e7666a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-2d655f77-05c9-4347-88a5-639d5e7666a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:53.982724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:53.983806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 20:08:53.984763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:53.987231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 221/437] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:53.988563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02460c46-b557-478b-b67f-90e8f8ade047 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.988983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02460c46-b557-478b-b67f-90e8f8ade047 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.989395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02460c46-b557-478b-b67f-90e8f8ade047 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.991073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 222/438] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:53.991390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865412b-8ed4-4422-8d3c-37dead02b1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:53.991998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865412b-8ed4-4422-8d3c-37dead02b1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:53.992328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865412b-8ed4-4422-8d3c-37dead02b1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:53.992552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865412b-8ed4-4422-8d3c-37dead02b1e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:53.993371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02460c46-b557-478b-b67f-90e8f8ade047 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:53.998489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865412b-8ed4-4422-8d3c-37dead02b1e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:53.998807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865412b-8ed4-4422-8d3c-37dead02b1e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:54.000223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865412b-8ed4-4422-8d3c-37dead02b1e6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:54.010333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 217/439] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:08:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:54.011552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9c915-282c-4d4c-90f8-431f8266e713 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.012003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9c915-282c-4d4c-90f8-431f8266e713 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.012278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9c915-282c-4d4c-90f8-431f8266e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.032800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f5d9c915-282c-4d4c-90f8-431f8266e713 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:54.034603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f5d9c915-282c-4d4c-90f8-431f8266e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:54.101576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:54.109423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 218/440] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:54.110719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9c915-282c-4d4c-90f8-431f8266e713 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.120298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02460c46-b557-478b-b67f-90e8f8ade047 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:54.158713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:54.164298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b045616-db11-4c35-996e-8533315204b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:54.168463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045616-db11-4c35-996e-8533315204b8 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.168463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045616-db11-4c35-996e-8533315204b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.168463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045616-db11-4c35-996e-8533315204b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.168463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b045616-db11-4c35-996e-8533315204b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:54.168463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b045616-db11-4c35-996e-8533315204b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:54.168463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b045616-db11-4c35-996e-8533315204b8 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:54.186507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 223/441] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:08:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:54.186507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b045616-db11-4c35-996e-8533315204b8 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.239807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9c915-282c-4d4c-90f8-431f8266e713 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:54.283594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbaaa2-6835-4730-9322-950a94b0d531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:54.284783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbaaa2-6835-4730-9322-950a94b0d531 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.284783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbaaa2-6835-4730-9322-950a94b0d531 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.284783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbaaa2-6835-4730-9322-950a94b0d531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.311231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b045616-db11-4c35-996e-8533315204b8 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:54.352267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dde1c-8ee8-4cc6-b964-334bf1cd5824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:54.352961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dde1c-8ee8-4cc6-b964-334bf1cd5824 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.353239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dde1c-8ee8-4cc6-b964-334bf1cd5824 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.353609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dde1c-8ee8-4cc6-b964-334bf1cd5824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.354521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbaaa2-6835-4730-9322-950a94b0d531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:54.355961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbaaa2-6835-4730-9322-950a94b0d531 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:54.372250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:54.395900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:54.424365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dde1c-8ee8-4cc6-b964-334bf1cd5824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:54.426765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dde1c-8ee8-4cc6-b964-334bf1cd5824 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:54.454783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 219/442] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:54.456088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbaaa2-6835-4730-9322-950a94b0d531 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.526433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 224/443] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 20:08:54.527195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dde1c-8ee8-4cc6-b964-334bf1cd5824 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.554139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbaaa2-6835-4730-9322-950a94b0d531 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:54.577262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:54.587917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc36f42-069c-4e09-b85f-64842962ea40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:54.588538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc36f42-069c-4e09-b85f-64842962ea40 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.588814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc36f42-069c-4e09-b85f-64842962ea40 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.589082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc36f42-069c-4e09-b85f-64842962ea40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.595832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:54.600032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc36f42-069c-4e09-b85f-64842962ea40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-edc36f42-069c-4e09-b85f-64842962ea40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:08:54.605160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:54.606120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:54.606896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 220/444] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:54.607519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc36f42-069c-4e09-b85f-64842962ea40 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.629500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dde1c-8ee8-4cc6-b964-334bf1cd5824 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:54.662707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7ec22-58f3-40b6-a8b0-bdde997bd26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:54.663330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7ec22-58f3-40b6-a8b0-bdde997bd26b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.663604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7ec22-58f3-40b6-a8b0-bdde997bd26b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.663880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7ec22-58f3-40b6-a8b0-bdde997bd26b None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.664987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7ec22-58f3-40b6-a8b0-bdde997bd26b None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:54.666453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7ec22-58f3-40b6-a8b0-bdde997bd26b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:54.668219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 225/445] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:54.668800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7ec22-58f3-40b6-a8b0-bdde997bd26b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.710728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc36f42-069c-4e09-b85f-64842962ea40 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:54.741926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309ce5b-e905-4b3d-ad66-c551322fb98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:54.742535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309ce5b-e905-4b3d-ad66-c551322fb98d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.742817 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309ce5b-e905-4b3d-ad66-c551322fb98d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.743082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309ce5b-e905-4b3d-ad66-c551322fb98d None admin] PATH_INFO: `/v3/projects/9c9189f4b1cc4920b3079cdc0d358919` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.744181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309ce5b-e905-4b3d-ad66-c551322fb98d None admin] RBAC: Authorizing `identity:get_project(project_id=9c9189f4b1cc4920b3079cdc0d358919)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:54.745657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309ce5b-e905-4b3d-ad66-c551322fb98d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:54.747246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 221/446] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/projects/9c9189f4b1cc4920b3079cdc0d358919 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:54.747992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309ce5b-e905-4b3d-ad66-c551322fb98d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.768068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7ec22-58f3-40b6-a8b0-bdde997bd26b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:54.784979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:54.803182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ab379-8894-42dc-8289-5f201bd3c601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:54.803781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab379-8894-42dc-8289-5f201bd3c601 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.804060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab379-8894-42dc-8289-5f201bd3c601 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.804331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab379-8894-42dc-8289-5f201bd3c601 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.804980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab379-8894-42dc-8289-5f201bd3c601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:54.805289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab379-8894-42dc-8289-5f201bd3c601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:54.806696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab379-8894-42dc-8289-5f201bd3c601 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:54.812012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:54.815674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 226/447] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:54.816464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ab379-8894-42dc-8289-5f201bd3c601 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.846664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309ce5b-e905-4b3d-ad66-c551322fb98d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:54.881035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0adfc-c1f0-4711-9db7-b4f3a20a0e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:54.881698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0adfc-c1f0-4711-9db7-b4f3a20a0e48 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.881983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0adfc-c1f0-4711-9db7-b4f3a20a0e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.882251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0adfc-c1f0-4711-9db7-b4f3a20a0e48 None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.883362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0adfc-c1f0-4711-9db7-b4f3a20a0e48 None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:54.884798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0adfc-c1f0-4711-9db7-b4f3a20a0e48 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:54.886568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 222/448] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:54.887142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0adfc-c1f0-4711-9db7-b4f3a20a0e48 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.920753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ab379-8894-42dc-8289-5f201bd3c601 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:54.954470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a75974-4b20-4352-9a32-f036e135180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:54.955116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a75974-4b20-4352-9a32-f036e135180e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:54.955388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a75974-4b20-4352-9a32-f036e135180e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:54.955656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a75974-4b20-4352-9a32-f036e135180e None admin] PATH_INFO: `/v3/groups/77ed0c80333e48488ea8194944fd70b1` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:54.956740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a75974-4b20-4352-9a32-f036e135180e None admin] RBAC: Authorizing `identity:get_group(group_id=77ed0c80333e48488ea8194944fd70b1)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:54.958232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a75974-4b20-4352-9a32-f036e135180e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:54.959788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 227/449] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/groups/77ed0c80333e48488ea8194944fd70b1 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:54.964860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a75974-4b20-4352-9a32-f036e135180e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:54.981903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0adfc-c1f0-4711-9db7-b4f3a20a0e48 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:55.012918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:55.015128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b9e58-21cf-4c73-ab66-1ca485ef363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:55.015718 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b9e58-21cf-4c73-ab66-1ca485ef363b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:55.015991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b9e58-21cf-4c73-ab66-1ca485ef363b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:55.016256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b9e58-21cf-4c73-ab66-1ca485ef363b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:55.018898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:55.064088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a75974-4b20-4352-9a32-f036e135180e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:55.082031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b9e58-21cf-4c73-ab66-1ca485ef363b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:55.083483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b9e58-21cf-4c73-ab66-1ca485ef363b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:55.099842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b951f4e-4808-40a3-9d97-af25fc6cb69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:55.100456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b951f4e-4808-40a3-9d97-af25fc6cb69f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:55.100727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b951f4e-4808-40a3-9d97-af25fc6cb69f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:55.100993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b951f4e-4808-40a3-9d97-af25fc6cb69f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:55.101732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b951f4e-4808-40a3-9d97-af25fc6cb69f None admin] RBAC: Adding query filter params (group.id=77ed0c80333e48488ea8194944fd70b1, role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:55.102034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b951f4e-4808-40a3-9d97-af25fc6cb69f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:55.103437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b951f4e-4808-40a3-9d97-af25fc6cb69f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:55.109097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 228/450] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/role_assignments?group.id=77ed0c80333e48488ea8194944fd70b1&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:55.109944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b951f4e-4808-40a3-9d97-af25fc6cb69f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:55.201845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 223/451] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:08:55.203077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b9e58-21cf-4c73-ab66-1ca485ef363b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:55.228518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b951f4e-4808-40a3-9d97-af25fc6cb69f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:55.240083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:55.244757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:55.270299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6ace7-583e-4d8f-ad2f-1e44928bbe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:55.270919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6ace7-583e-4d8f-ad2f-1e44928bbe9d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:55.271195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6ace7-583e-4d8f-ad2f-1e44928bbe9d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:55.271465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6ace7-583e-4d8f-ad2f-1e44928bbe9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:55.272193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6ace7-583e-4d8f-ad2f-1e44928bbe9d None admin] RBAC: Adding query filter params (group.id=77ed0c80333e48488ea8194944fd70b1, role.id=84d6c85b41034cae925afeb8921381e8, scope.project.id=9c9189f4b1cc4920b3079cdc0d358919) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:55.272495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6ace7-583e-4d8f-ad2f-1e44928bbe9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:55.273965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6ace7-583e-4d8f-ad2f-1e44928bbe9d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:55.280606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 229/452] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:55 2022] GET /identity/v3/role_assignments?group.id=77ed0c80333e48488ea8194944fd70b1&scope.project.id=9c9189f4b1cc4920b3079cdc0d358919&role.id=84d6c85b41034cae925afeb8921381e8 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:55.322151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b9e58-21cf-4c73-ab66-1ca485ef363b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:55.362683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d9a6-030b-4dc8-91d7-de2a6e53a935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:55.363327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d9a6-030b-4dc8-91d7-de2a6e53a935 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:55.363673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d9a6-030b-4dc8-91d7-de2a6e53a935 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:55.363907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593d9a6-030b-4dc8-91d7-de2a6e53a935 None admin] PATH_INFO: `/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:55.365047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593d9a6-030b-4dc8-91d7-de2a6e53a935 None admin] RBAC: Authorizing `identity:get_project(project_id=e6d76b1c203e49b8ae5f7f394a20a597)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:55.366544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593d9a6-030b-4dc8-91d7-de2a6e53a935 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:55.368211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 224/453] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:08:55 2022] GET /identity/v3/projects/e6d76b1c203e49b8ae5f7f394a20a597 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:55.373933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6ace7-583e-4d8f-ad2f-1e44928bbe9d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:55.476396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6ace7-583e-4d8f-ad2f-1e44928bbe9d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:55.509597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3293157-9679-4686-94fb-c83f35132674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:55.510224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3293157-9679-4686-94fb-c83f35132674 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:55.510503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3293157-9679-4686-94fb-c83f35132674 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:55.510782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3293157-9679-4686-94fb-c83f35132674 None admin] PATH_INFO: `/v3/groups/8eea4665d08047158fbd6ca8077179cb` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:55.511911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3293157-9679-4686-94fb-c83f35132674 None admin] RBAC: Authorizing `identity:get_group(group_id=8eea4665d08047158fbd6ca8077179cb)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:55.513346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3293157-9679-4686-94fb-c83f35132674 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:55.514991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 230/454] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 20:08:55 2022] GET /identity/v3/groups/8eea4665d08047158fbd6ca8077179cb => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:55.521967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d9a6-030b-4dc8-91d7-de2a6e53a935 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:55.627295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593d9a6-030b-4dc8-91d7-de2a6e53a935 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:55.628833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:55.663341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44963dc-8ef9-4eae-907c-cbafdd58c8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:55.663948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44963dc-8ef9-4eae-907c-cbafdd58c8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:55.664226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44963dc-8ef9-4eae-907c-cbafdd58c8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:55.664498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44963dc-8ef9-4eae-907c-cbafdd58c8d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:55.665225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44963dc-8ef9-4eae-907c-cbafdd58c8d2 None admin] RBAC: Adding query filter params (group.id=8eea4665d08047158fbd6ca8077179cb, role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=e6d76b1c203e49b8ae5f7f394a20a597) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:55.665546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44963dc-8ef9-4eae-907c-cbafdd58c8d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:55.666979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44963dc-8ef9-4eae-907c-cbafdd58c8d2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:55.673080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 225/455] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 20:08:55 2022] GET /identity/v3/role_assignments?group.id=8eea4665d08047158fbd6ca8077179cb&scope.project.id=e6d76b1c203e49b8ae5f7f394a20a597&role.id=36331ec562ff42f883fc883363bc4c8d => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:58.143093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fd0e9-7c06-48cb-a5e1-fd4d44bbad1e None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:58.143575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fd0e9-7c06-48cb-a5e1-fd4d44bbad1e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:58.143575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fd0e9-7c06-48cb-a5e1-fd4d44bbad1e None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:58.145082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 231/456] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:58.150839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ec59a-6709-4d95-b766-3ca6e862732a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:58.151268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ec59a-6709-4d95-b766-3ca6e862732a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:58.151446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ec59a-6709-4d95-b766-3ca6e862732a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:58.178499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-799ec59a-6709-4d95-b766-3ca6e862732a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:58.179699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-799ec59a-6709-4d95-b766-3ca6e862732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:58.271366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 226/457] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:08:58.598956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fd0e9-7c06-48cb-a5e1-fd4d44bbad1e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:58.622751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe0deb-4402-45ef-8c0f-e06c9d76fdc0 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:58.624480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe0deb-4402-45ef-8c0f-e06c9d76fdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:58.625835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe0deb-4402-45ef-8c0f-e06c9d76fdc0 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:58.630760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 227/458] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:58.637483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345c881-98ff-4620-89d4-28e1b42ff0f5 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:58.637802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345c881-98ff-4620-89d4-28e1b42ff0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:58.638055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345c881-98ff-4620-89d4-28e1b42ff0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:58.656703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-0345c881-98ff-4620-89d4-28e1b42ff0f5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:58.658248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0345c881-98ff-4620-89d4-28e1b42ff0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:58.664355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:58.730721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fd0e9-7c06-48cb-a5e1-fd4d44bbad1e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:58.743403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 228/459] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:58.744943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebc606-8098-4635-9034-0a00ab0cfdd2 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:58.745426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebc606-8098-4635-9034-0a00ab0cfdd2 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:58.745934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebc606-8098-4635-9034-0a00ab0cfdd2 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:58.747385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 229/460] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:58.753359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ec13d-5e19-452f-9c9b-aed49ea2fa1e None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:58.753901 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ec13d-5e19-452f-9c9b-aed49ea2fa1e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:58.754430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ec13d-5e19-452f-9c9b-aed49ea2fa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:58.785251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-729ec13d-5e19-452f-9c9b-aed49ea2fa1e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:58.786468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-729ec13d-5e19-452f-9c9b-aed49ea2fa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:58.807941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d77e1-65c1-4acf-99b7-31a433767908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:58.808580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d77e1-65c1-4acf-99b7-31a433767908 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:58.808858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d77e1-65c1-4acf-99b7-31a433767908 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:58.809129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d77e1-65c1-4acf-99b7-31a433767908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:58.829551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d77e1-65c1-4acf-99b7-31a433767908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:58.830377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:58.831564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d77e1-65c1-4acf-99b7-31a433767908 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-ff2d77e1-65c1-4acf-99b7-31a433767908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:08:58.858456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:58.859387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:08:58.860201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:08:58.860201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:08:58.860201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:08:58.860201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 232/461] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:58.870654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d77e1-65c1-4acf-99b7-31a433767908 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:58.935237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 230/462] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:58.988273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:58.995471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d77e1-65c1-4acf-99b7-31a433767908 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:59.038920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ec13d-5e19-452f-9c9b-aed49ea2fa1e None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:59.040847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98f976-bfee-45a6-b4f7-c21961004594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:59.041448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98f976-bfee-45a6-b4f7-c21961004594 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.041762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98f976-bfee-45a6-b4f7-c21961004594 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.042191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98f976-bfee-45a6-b4f7-c21961004594 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.042853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98f976-bfee-45a6-b4f7-c21961004594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:59.043156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98f976-bfee-45a6-b4f7-c21961004594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:59.045371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98f976-bfee-45a6-b4f7-c21961004594 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:59.060812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 233/463] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:59.067327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98f976-bfee-45a6-b4f7-c21961004594 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:59.201175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ec13d-5e19-452f-9c9b-aed49ea2fa1e None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:59.242762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf1238-6c97-4200-a897-33dd7149c5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:59.243887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf1238-6c97-4200-a897-33dd7149c5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.244387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf1238-6c97-4200-a897-33dd7149c5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.244986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf1238-6c97-4200-a897-33dd7149c5c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.256827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf1238-6c97-4200-a897-33dd7149c5c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:59.258644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf1238-6c97-4200-a897-33dd7149c5c9 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-52bf1238-6c97-4200-a897-33dd7149c5c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:08:59.276630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:59.277583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:08:59.278433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:08:59.278433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:08:59.278433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:59.280486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 231/464] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:08:59.281433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98f976-bfee-45a6-b4f7-c21961004594 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:59.282030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf1238-6c97-4200-a897-33dd7149c5c9 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:59.294686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:59.325874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1cc3d-0f4b-47eb-b253-b6d3ac773cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:59.326977 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1cc3d-0f4b-47eb-b253-b6d3ac773cd8 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.327253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1cc3d-0f4b-47eb-b253-b6d3ac773cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.327517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1cc3d-0f4b-47eb-b253-b6d3ac773cd8 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.328594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1cc3d-0f4b-47eb-b253-b6d3ac773cd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:59.330524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1cc3d-0f4b-47eb-b253-b6d3ac773cd8 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:59.361542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 234/465] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:08:59 2022] POST /identity/v3/users => generated 246 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:08:59.361542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1cc3d-0f4b-47eb-b253-b6d3ac773cd8 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:59.383562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:59.402792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf1238-6c97-4200-a897-33dd7149c5c9 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:59.443827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b43073-aa9c-44f4-acaa-747d702974f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:59.444649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b43073-aa9c-44f4-acaa-747d702974f6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.445006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b43073-aa9c-44f4-acaa-747d702974f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.445368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b43073-aa9c-44f4-acaa-747d702974f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.457769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b43073-aa9c-44f4-acaa-747d702974f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:59.459328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b43073-aa9c-44f4-acaa-747d702974f6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-32b43073-aa9c-44f4-acaa-747d702974f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:08:59.475652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:08:59.476584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:08:59.477642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:08:59.477642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:08:59.477642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:08:59.477642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 232/466] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:08:59.480361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e3138-ce59-418f-b4ca-0e9500ba0e2e None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.480361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e3138-ce59-418f-b4ca-0e9500ba0e2e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.480361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e3138-ce59-418f-b4ca-0e9500ba0e2e None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.480778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 233/467] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:59.482570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e3138-ce59-418f-b4ca-0e9500ba0e2e None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:59.501193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1cc3d-0f4b-47eb-b253-b6d3ac773cd8 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:59.547717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b1fe2-4df0-4171-85f0-56dfad18441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:59.548323 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b1fe2-4df0-4171-85f0-56dfad18441a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.549185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b1fe2-4df0-4171-85f0-56dfad18441a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.549461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b1fe2-4df0-4171-85f0-56dfad18441a None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.550449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b1fe2-4df0-4171-85f0-56dfad18441a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:59.550759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b1fe2-4df0-4171-85f0-56dfad18441a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:59.552681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b1fe2-4df0-4171-85f0-56dfad18441a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:59.562662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:59.569608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 235/468] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:08:59.571190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab4fb-8467-4758-a3de-41b49877950c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.571190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab4fb-8467-4758-a3de-41b49877950c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.571542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ab4fb-8467-4758-a3de-41b49877950c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.591230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-414ab4fb-8467-4758-a3de-41b49877950c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:59.592429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-414ab4fb-8467-4758-a3de-41b49877950c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:08:59.622828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e3138-ce59-418f-b4ca-0e9500ba0e2e None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:59.661608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac9256-444e-4f5a-85e9-198141fec57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:59.662218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac9256-444e-4f5a-85e9-198141fec57d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.662503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac9256-444e-4f5a-85e9-198141fec57d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.662775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac9256-444e-4f5a-85e9-198141fec57d None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.663418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac9256-444e-4f5a-85e9-198141fec57d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:08:59.663719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac9256-444e-4f5a-85e9-198141fec57d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:59.665085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac9256-444e-4f5a-85e9-198141fec57d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:59.674508 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 236/469] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:08:59.675502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ab4fb-8467-4758-a3de-41b49877950c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:59.683531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 234/470] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:08:59.684432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:59.689022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac9256-444e-4f5a-85e9-198141fec57d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:08:59.817273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac9256-444e-4f5a-85e9-198141fec57d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:59.823764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ab4fb-8467-4758-a3de-41b49877950c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:08:59.841635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:08:59.859458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02b9f9-e7c2-4654-99b7-71108ae4d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:59.860398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02b9f9-e7c2-4654-99b7-71108ae4d350 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.860863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02b9f9-e7c2-4654-99b7-71108ae4d350 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.861346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02b9f9-e7c2-4654-99b7-71108ae4d350 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.862682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02b9f9-e7c2-4654-99b7-71108ae4d350 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:59.865244 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02b9f9-e7c2-4654-99b7-71108ae4d350 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:59.896492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f104a2-3b0e-4e02-948f-2567c2141a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:08:59.897888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f104a2-3b0e-4e02-948f-2567c2141a29 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.901918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f104a2-3b0e-4e02-948f-2567c2141a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.905292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 235/471] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:08:59 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:08:59.905818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f104a2-3b0e-4e02-948f-2567c2141a29 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.906683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f104a2-3b0e-4e02-948f-2567c2141a29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:08:59.908421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f104a2-3b0e-4e02-948f-2567c2141a29 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:08:59.933224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 237/472] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:08:59 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:08:59.943212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb766d-fe0c-4f64-bafd-909b26b9fe44 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.943531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb766d-fe0c-4f64-bafd-909b26b9fe44 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.943774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb766d-fe0c-4f64-bafd-909b26b9fe44 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.945565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 236/473] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:08:59.951571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cf776-f3cc-4ad2-8169-44da10b0dbd7 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:08:59.951877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cf776-f3cc-4ad2-8169-44da10b0dbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:08:59.952128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cf776-f3cc-4ad2-8169-44da10b0dbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:08:59.973252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-5b8cf776-f3cc-4ad2-8169-44da10b0dbd7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:08:59.974837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5b8cf776-f3cc-4ad2-8169-44da10b0dbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:00.038202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:00.040027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb766d-fe0c-4f64-bafd-909b26b9fe44 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:00.055839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 238/474] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:00.157644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb766d-fe0c-4f64-bafd-909b26b9fe44 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:00.198907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd78d7-7d6d-46b2-b149-04d3b04d21bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:00.199540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd78d7-7d6d-46b2-b149-04d3b04d21bb None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:00.199814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd78d7-7d6d-46b2-b149-04d3b04d21bb None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:00.200080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd78d7-7d6d-46b2-b149-04d3b04d21bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:00.211963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd78d7-7d6d-46b2-b149-04d3b04d21bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:00.213439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd78d7-7d6d-46b2-b149-04d3b04d21bb None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-02fd78d7-7d6d-46b2-b149-04d3b04d21bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:00.230930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:00.231844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:00.232641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:00.232641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:00.232641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:00.235208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 237/475] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:00.238117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cf776-f3cc-4ad2-8169-44da10b0dbd7 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:00.342929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cf776-f3cc-4ad2-8169-44da10b0dbd7 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:00.343520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd78d7-7d6d-46b2-b149-04d3b04d21bb None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:00.381996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:00.390755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a99b8-3107-4676-9876-887750d44c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:00.391365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a99b8-3107-4676-9876-887750d44c10 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:00.391638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a99b8-3107-4676-9876-887750d44c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:00.392008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a99b8-3107-4676-9876-887750d44c10 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:00.393210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a99b8-3107-4676-9876-887750d44c10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:00.393592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a99b8-3107-4676-9876-887750d44c10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:00.394922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a99b8-3107-4676-9876-887750d44c10 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:00.407588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 239/476] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:00.414266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a99b8-3107-4676-9876-887750d44c10 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:00.475620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd78d7-7d6d-46b2-b149-04d3b04d21bb None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:00.480487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:00.520958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:00.523285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:00.523777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:00.525908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:00.526944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:00.528579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:00.536368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.sql.core [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 05 20:09:00.536368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 05 20:09:00.536368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [parameters: {'id': 'e329ab5a7e5a49bcb0f0174782e7ccaf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 05 20:09:00.536368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81338) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 05 20:09:00.544342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a99b8-3107-4676-9876-887750d44c10 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application conn.query(q) Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application result.read() Oct 05 20:09:00.560226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return ref.to_dict() Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 20:09:00.561131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application next(self.gen) Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application yield resource Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application next(self.gen) Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.session.rollback() Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.force_reraise() Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise self.value Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application session.commit() Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 05 20:09:00.562050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._prepare_impl() Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.session.flush() Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._flush(objects) Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application compat.raise_( Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application flush_context.execute() Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rec.execute(self) Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 05 20:09:00.562881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 20:09:00.563716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application conn.query(q) Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application result.read() Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 20:09:00.565317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [parameters: {'id': 'e329ab5a7e5a49bcb0f0174782e7ccaf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:00.566105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 05 20:09:00.566949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:00.576494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 238/477] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 20:09:00 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:00.580996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892116bd-40b7-4fe8-8440-12b8e354e4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:00.581623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892116bd-40b7-4fe8-8440-12b8e354e4b2 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:00.581901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892116bd-40b7-4fe8-8440-12b8e354e4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:00.582169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892116bd-40b7-4fe8-8440-12b8e354e4b2 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:00.582803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892116bd-40b7-4fe8-8440-12b8e354e4b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:00.584327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:00.584611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892116bd-40b7-4fe8-8440-12b8e354e4b2 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:00.611186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 240/478] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:09:00 2022] POST /identity/v3/users => generated 249 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:09:00.734766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:00.745150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c3146-2839-4be6-82ba-d1ff9e3c09ee None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:00.791770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb8029-95eb-48ff-8a6c-5cfe4154631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:00.792794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb8029-95eb-48ff-8a6c-5cfe4154631b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:00.793305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb8029-95eb-48ff-8a6c-5cfe4154631b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:00.793862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb8029-95eb-48ff-8a6c-5cfe4154631b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:00.806064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb8029-95eb-48ff-8a6c-5cfe4154631b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-b6fb8029-95eb-48ff-8a6c-5cfe4154631b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:00.811316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:00.812193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 20:09:00.812937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:00.816129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 239/479] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:00.821650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892116bd-40b7-4fe8-8440-12b8e354e4b2 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:00.937688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892116bd-40b7-4fe8-8440-12b8e354e4b2 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:00.941053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:00.974872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf0668-b7c9-4772-9c12-8263cb2872fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:00.975623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf0668-b7c9-4772-9c12-8263cb2872fe None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:00.976005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf0668-b7c9-4772-9c12-8263cb2872fe None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:00.976431 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf0668-b7c9-4772-9c12-8263cb2872fe None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:00.977202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf0668-b7c9-4772-9c12-8263cb2872fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:00.977636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf0668-b7c9-4772-9c12-8263cb2872fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:00.979216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf0668-b7c9-4772-9c12-8263cb2872fe None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:00.988924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 241/480] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:02.060143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fac152-1131-45ad-b7d5-2e6f22f84c08 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:02.060664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fac152-1131-45ad-b7d5-2e6f22f84c08 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:02.060946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fac152-1131-45ad-b7d5-2e6f22f84c08 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:02.062572 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 240/481] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:02.069054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854ff13-72e8-49cd-9218-4b0e673a0663 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:02.069603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854ff13-72e8-49cd-9218-4b0e673a0663 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:02.069692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854ff13-72e8-49cd-9218-4b0e673a0663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:02.098968 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-0854ff13-72e8-49cd-9218-4b0e673a0663 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:02.100159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0854ff13-72e8-49cd-9218-4b0e673a0663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:02.174531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 242/482] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:09:02.491794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fac152-1131-45ad-b7d5-2e6f22f84c08 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:02.626506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fac152-1131-45ad-b7d5-2e6f22f84c08 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:02.678221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a09a33-1ccd-43f2-a42a-62246e1697e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:02.678862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a09a33-1ccd-43f2-a42a-62246e1697e7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:02.679146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a09a33-1ccd-43f2-a42a-62246e1697e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:02.679416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a09a33-1ccd-43f2-a42a-62246e1697e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:02.694407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a09a33-1ccd-43f2-a42a-62246e1697e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-b5a09a33-1ccd-43f2-a42a-62246e1697e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:02.699648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:02.700506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:02.701269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 241/483] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:02.707865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854ff13-72e8-49cd-9218-4b0e673a0663 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:02.838221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:02.846787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854ff13-72e8-49cd-9218-4b0e673a0663 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:02.884034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7f67d-d7d1-49c9-837b-c57717698cab None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:02.884314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7f67d-d7d1-49c9-837b-c57717698cab None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:02.884589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7f67d-d7d1-49c9-837b-c57717698cab None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:02.886821 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 242/484] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:02.891853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371ae1-82d9-4d07-9a97-5f194cddf987 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:02.892160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371ae1-82d9-4d07-9a97-5f194cddf987 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:02.892441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62371ae1-82d9-4d07-9a97-5f194cddf987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:02.899029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba58c81-85b8-4c9c-b377-b2e3735ce573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:02.899895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba58c81-85b8-4c9c-b377-b2e3735ce573 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:02.900365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba58c81-85b8-4c9c-b377-b2e3735ce573 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:02.900826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba58c81-85b8-4c9c-b377-b2e3735ce573 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:02.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba58c81-85b8-4c9c-b377-b2e3735ce573 None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:02.902206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba58c81-85b8-4c9c-b377-b2e3735ce573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:02.903768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba58c81-85b8-4c9c-b377-b2e3735ce573 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:02.919334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 243/485] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:02.925555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba58c81-85b8-4c9c-b377-b2e3735ce573 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:02.929812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-62371ae1-82d9-4d07-9a97-5f194cddf987 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:02.935882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-62371ae1-82d9-4d07-9a97-5f194cddf987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:02.947269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:03.037743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 243/486] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:03.039208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9a675-d509-410d-8438-37b024ca17d6 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.039606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9a675-d509-410d-8438-37b024ca17d6 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.039884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9a675-d509-410d-8438-37b024ca17d6 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.045432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 244/487] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:03.047027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e7b2d-cfbf-413b-907a-8ffbf99b215d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.047385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e7b2d-cfbf-413b-907a-8ffbf99b215d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.047666 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e7b2d-cfbf-413b-907a-8ffbf99b215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.064754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba58c81-85b8-4c9c-b377-b2e3735ce573 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:03.066777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-746e7b2d-cfbf-413b-907a-8ffbf99b215d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:03.068065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-746e7b2d-cfbf-413b-907a-8ffbf99b215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:03.108355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055a76e-bc34-4fef-999f-983879be1e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:03.108966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055a76e-bc34-4fef-999f-983879be1e80 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.113532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055a76e-bc34-4fef-999f-983879be1e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.113532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055a76e-bc34-4fef-999f-983879be1e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.133209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055a76e-bc34-4fef-999f-983879be1e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:03.135174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055a76e-bc34-4fef-999f-983879be1e80 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-c055a76e-bc34-4fef-999f-983879be1e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:03.161604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:03.162534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:03.163338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:03.163338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:03.163338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:03.163338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 244/488] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:03.168467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055a76e-bc34-4fef-999f-983879be1e80 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:03.186434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 245/489] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:09:03.191575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:03.292201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055a76e-bc34-4fef-999f-983879be1e80 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:03.336123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4406d-bd20-48a6-89c9-d7cbf60ed6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:03.336724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4406d-bd20-48a6-89c9-d7cbf60ed6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.336994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4406d-bd20-48a6-89c9-d7cbf60ed6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.337294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4406d-bd20-48a6-89c9-d7cbf60ed6d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.337960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4406d-bd20-48a6-89c9-d7cbf60ed6d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:03.338253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4406d-bd20-48a6-89c9-d7cbf60ed6d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:03.339672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4406d-bd20-48a6-89c9-d7cbf60ed6d1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:03.359506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 245/490] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:03.365598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e7b2d-cfbf-413b-907a-8ffbf99b215d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:03.463408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:03.496765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e7b2d-cfbf-413b-907a-8ffbf99b215d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:03.531279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6ea83-33b1-4806-a27f-aab7f84b6244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:03.531912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6ea83-33b1-4806-a27f-aab7f84b6244 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.532186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6ea83-33b1-4806-a27f-aab7f84b6244 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.532452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f6ea83-33b1-4806-a27f-aab7f84b6244 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.540181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4406d-bd20-48a6-89c9-d7cbf60ed6d1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:03.543100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6ea83-33b1-4806-a27f-aab7f84b6244 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:03.544657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f6ea83-33b1-4806-a27f-aab7f84b6244 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-38f6ea83-33b1-4806-a27f-aab7f84b6244 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:03.565705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:03.566635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:03.567529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:03.573544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 246/491] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 20:09:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:03.574447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6ea83-33b1-4806-a27f-aab7f84b6244 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:03.646510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:03.678948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f6ea83-33b1-4806-a27f-aab7f84b6244 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:03.682411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4406d-bd20-48a6-89c9-d7cbf60ed6d1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:03.713653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:03.719229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3d354-c6b0-488d-92cb-81e9f81405df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:03.720865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3d354-c6b0-488d-92cb-81e9f81405df None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.721139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3d354-c6b0-488d-92cb-81e9f81405df None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.721407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3d354-c6b0-488d-92cb-81e9f81405df None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.722161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3d354-c6b0-488d-92cb-81e9f81405df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:03.722466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3d354-c6b0-488d-92cb-81e9f81405df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:03.723530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350071bd-34cc-400e-b59c-fd99cad98704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:03.723971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3d354-c6b0-488d-92cb-81e9f81405df None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:03.724161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350071bd-34cc-400e-b59c-fd99cad98704 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.724438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350071bd-34cc-400e-b59c-fd99cad98704 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.724708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350071bd-34cc-400e-b59c-fd99cad98704 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.734895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350071bd-34cc-400e-b59c-fd99cad98704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-350071bd-34cc-400e-b59c-fd99cad98704 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:03.741572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:03.742412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:03.743169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:03.747727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 246/492] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:03.748739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cef64-ee73-4b15-82a9-cb3bb8be4aa2 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.749025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cef64-ee73-4b15-82a9-cb3bb8be4aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.749302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cef64-ee73-4b15-82a9-cb3bb8be4aa2 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.753644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 247/493] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:03.753644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cef64-ee73-4b15-82a9-cb3bb8be4aa2 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:03.762739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 247/494] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 20:09:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:03.763580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3d354-c6b0-488d-92cb-81e9f81405df None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:03.861658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cef64-ee73-4b15-82a9-cb3bb8be4aa2 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:03.870938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3d354-c6b0-488d-92cb-81e9f81405df None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:03.890916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:03.897615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63305d4e-30b3-4a5d-a2f0-64b1b4fb94dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:03.898213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63305d4e-30b3-4a5d-a2f0-64b1b4fb94dd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.898491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63305d4e-30b3-4a5d-a2f0-64b1b4fb94dd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.898759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63305d4e-30b3-4a5d-a2f0-64b1b4fb94dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.904892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94604d8-69f8-4195-b172-397091e354b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:03.905487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94604d8-69f8-4195-b172-397091e354b9 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.905788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94604d8-69f8-4195-b172-397091e354b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.906050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94604d8-69f8-4195-b172-397091e354b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.906702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94604d8-69f8-4195-b172-397091e354b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:03.906994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94604d8-69f8-4195-b172-397091e354b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:03.908946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94604d8-69f8-4195-b172-397091e354b9 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:03.909728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63305d4e-30b3-4a5d-a2f0-64b1b4fb94dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-63305d4e-30b3-4a5d-a2f0-64b1b4fb94dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:03.914421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:03.915277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:03.916045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 248/495] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:03.916683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c84c1-fa44-4842-98d2-0d79d4ea937c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:03.917203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c84c1-fa44-4842-98d2-0d79d4ea937c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:03.917293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 248/496] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:03.917368 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c84c1-fa44-4842-98d2-0d79d4ea937c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:03.921406 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94604d8-69f8-4195-b172-397091e354b9 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:03.935482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7e3c84c1-fa44-4842-98d2-0d79d4ea937c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:03.937149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7e3c84c1-fa44-4842-98d2-0d79d4ea937c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:03.972286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:04.002379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 249/497] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:04.003598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1cc40-33cc-49dc-b30e-1d02ae8460d7 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.003880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1cc40-33cc-49dc-b30e-1d02ae8460d7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.004154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1cc40-33cc-49dc-b30e-1d02ae8460d7 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.005375 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 250/498] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:04.006096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1cc40-33cc-49dc-b30e-1d02ae8460d7 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.031270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94604d8-69f8-4195-b172-397091e354b9 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.068629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d9069-e0a3-4175-9b3b-82577917a0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.069219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9069-e0a3-4175-9b3b-82577917a0ee None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.069478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9069-e0a3-4175-9b3b-82577917a0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.069844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d9069-e0a3-4175-9b3b-82577917a0ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.115944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1cc40-33cc-49dc-b30e-1d02ae8460d7 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.138404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:04.144337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d9069-e0a3-4175-9b3b-82577917a0ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:04.145843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d9069-e0a3-4175-9b3b-82577917a0ee None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:04.156362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31a06c-9ba8-470a-8768-4636f5a57c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.157244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31a06c-9ba8-470a-8768-4636f5a57c7e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.158268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31a06c-9ba8-470a-8768-4636f5a57c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.158595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31a06c-9ba8-470a-8768-4636f5a57c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.170103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31a06c-9ba8-470a-8768-4636f5a57c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:04.171524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca31a06c-9ba8-470a-8768-4636f5a57c7e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-ca31a06c-9ba8-470a-8768-4636f5a57c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:04.186566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:04.187441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:04.188277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:04.188277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:04.188277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:04.188277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 251/499] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:04.188481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31a06c-9ba8-470a-8768-4636f5a57c7e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.205784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:04.257456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 249/500] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:09:04.258725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbecd3-06bd-4bdd-ab3a-dbaaf7030cea None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.259006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbecd3-06bd-4bdd-ab3a-dbaaf7030cea None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.259273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbecd3-06bd-4bdd-ab3a-dbaaf7030cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.277368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8dcbecd3-06bd-4bdd-ab3a-dbaaf7030cea None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:04.278595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8dcbecd3-06bd-4bdd-ab3a-dbaaf7030cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:04.299471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31a06c-9ba8-470a-8768-4636f5a57c7e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.332124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078a0cf-ca90-49e3-a455-db270027e6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.332722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078a0cf-ca90-49e3-a455-db270027e6cd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.332995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078a0cf-ca90-49e3-a455-db270027e6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.333258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078a0cf-ca90-49e3-a455-db270027e6cd None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.333957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078a0cf-ca90-49e3-a455-db270027e6cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:04.334258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078a0cf-ca90-49e3-a455-db270027e6cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:04.335607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078a0cf-ca90-49e3-a455-db270027e6cd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:04.343132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 250/501] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:04.343268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 252/502] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:04.343943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbecd3-06bd-4bdd-ab3a-dbaaf7030cea None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.344040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078a0cf-ca90-49e3-a455-db270027e6cd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.390188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:04.441524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:04.461319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078a0cf-ca90-49e3-a455-db270027e6cd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.461497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbecd3-06bd-4bdd-ab3a-dbaaf7030cea None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.495014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041e6dc-324c-4058-a394-7bc046b6cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.495609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041e6dc-324c-4058-a394-7bc046b6cf5a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.495885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041e6dc-324c-4058-a394-7bc046b6cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.496154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041e6dc-324c-4058-a394-7bc046b6cf5a None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.496792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041e6dc-324c-4058-a394-7bc046b6cf5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:04.497197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041e6dc-324c-4058-a394-7bc046b6cf5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:04.498597 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041e6dc-324c-4058-a394-7bc046b6cf5a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:04.499322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67902392-5b88-4df7-abe1-0a27dcaa28f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.500228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67902392-5b88-4df7-abe1-0a27dcaa28f4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.500777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67902392-5b88-4df7-abe1-0a27dcaa28f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.501055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67902392-5b88-4df7-abe1-0a27dcaa28f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.509590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 251/503] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:04.510381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041e6dc-324c-4058-a394-7bc046b6cf5a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.516812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67902392-5b88-4df7-abe1-0a27dcaa28f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:04.519160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67902392-5b88-4df7-abe1-0a27dcaa28f4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-67902392-5b88-4df7-abe1-0a27dcaa28f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:04.534632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:04.535515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:04.536321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:04.536321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:04.536321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:04.540600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 253/504] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:04.541422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67902392-5b88-4df7-abe1-0a27dcaa28f4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.616920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041e6dc-324c-4058-a394-7bc046b6cf5a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.632016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:04.648124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67902392-5b88-4df7-abe1-0a27dcaa28f4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.653048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca060a-8ae2-44e1-ae7e-a12ddc9a4831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.653712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca060a-8ae2-44e1-ae7e-a12ddc9a4831 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.654351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca060a-8ae2-44e1-ae7e-a12ddc9a4831 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.654351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca060a-8ae2-44e1-ae7e-a12ddc9a4831 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.664779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca060a-8ae2-44e1-ae7e-a12ddc9a4831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-adca060a-8ae2-44e1-ae7e-a12ddc9a4831 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:04.670538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:04.671395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:04.672166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 252/505] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:04.673815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca060a-8ae2-44e1-ae7e-a12ddc9a4831 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.676375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:04.684331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff14cb-2de6-4f17-a70d-f4e6cba2c296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.684960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff14cb-2de6-4f17-a70d-f4e6cba2c296 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.685242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff14cb-2de6-4f17-a70d-f4e6cba2c296 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.685537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff14cb-2de6-4f17-a70d-f4e6cba2c296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.697413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff14cb-2de6-4f17-a70d-f4e6cba2c296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:04.699087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff14cb-2de6-4f17-a70d-f4e6cba2c296 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-20ff14cb-2de6-4f17-a70d-f4e6cba2c296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:04.714937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:04.716008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:04.717057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:04.717057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:04.717057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:04.717057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 254/506] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:04.717873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff14cb-2de6-4f17-a70d-f4e6cba2c296 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.774825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca060a-8ae2-44e1-ae7e-a12ddc9a4831 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.807880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48527063-4549-4102-bc73-53e67e669077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.808502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48527063-4549-4102-bc73-53e67e669077 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.808781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48527063-4549-4102-bc73-53e67e669077 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.809053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48527063-4549-4102-bc73-53e67e669077 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.815563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff14cb-2de6-4f17-a70d-f4e6cba2c296 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.816667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48527063-4549-4102-bc73-53e67e669077 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:04.818217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48527063-4549-4102-bc73-53e67e669077 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-48527063-4549-4102-bc73-53e67e669077 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:04.829113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:04.830531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 253/507] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:04.831453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48527063-4549-4102-bc73-53e67e669077 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.854317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18960f3e-2bf1-4d5a-b2b9-5153ef191f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.854912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18960f3e-2bf1-4d5a-b2b9-5153ef191f05 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.855188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18960f3e-2bf1-4d5a-b2b9-5153ef191f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.855455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18960f3e-2bf1-4d5a-b2b9-5153ef191f05 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.856083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18960f3e-2bf1-4d5a-b2b9-5153ef191f05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:04.856378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18960f3e-2bf1-4d5a-b2b9-5153ef191f05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:04.857741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18960f3e-2bf1-4d5a-b2b9-5153ef191f05 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:04.864639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:04.870516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 255/508] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:04.871540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18960f3e-2bf1-4d5a-b2b9-5153ef191f05 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:04.895195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:04.934286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48527063-4549-4102-bc73-53e67e669077 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.967682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad1e0e-fa3c-4c77-98bb-325e124fdebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:04.968235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad1e0e-fa3c-4c77-98bb-325e124fdebe None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:04.968506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad1e0e-fa3c-4c77-98bb-325e124fdebe None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:04.968774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad1e0e-fa3c-4c77-98bb-325e124fdebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:04.973768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18960f3e-2bf1-4d5a-b2b9-5153ef191f05 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:04.979036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad1e0e-fa3c-4c77-98bb-325e124fdebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:04.980476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad1e0e-fa3c-4c77-98bb-325e124fdebe None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-44ad1e0e-fa3c-4c77-98bb-325e124fdebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:04.993995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:04.995049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:04.996092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:04.996092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:04.996092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:04.996092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 254/509] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:04.996092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad1e0e-fa3c-4c77-98bb-325e124fdebe None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.011196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfe6d9-37a3-4282-8c51-9ef5dc32775f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.011803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfe6d9-37a3-4282-8c51-9ef5dc32775f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.012080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfe6d9-37a3-4282-8c51-9ef5dc32775f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.012354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfe6d9-37a3-4282-8c51-9ef5dc32775f None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.013012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfe6d9-37a3-4282-8c51-9ef5dc32775f None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:05.013314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfe6d9-37a3-4282-8c51-9ef5dc32775f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.014712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bfe6d9-37a3-4282-8c51-9ef5dc32775f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.023434 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 256/510] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:05.024201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfe6d9-37a3-4282-8c51-9ef5dc32775f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.082783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:05.096172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad1e0e-fa3c-4c77-98bb-325e124fdebe None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.109084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:05.122926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bfe6d9-37a3-4282-8c51-9ef5dc32775f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.128823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea992bb-c5d5-4919-8821-4c456f899434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.129410 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea992bb-c5d5-4919-8821-4c456f899434 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.129717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea992bb-c5d5-4919-8821-4c456f899434 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.129988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea992bb-c5d5-4919-8821-4c456f899434 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.130609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea992bb-c5d5-4919-8821-4c456f899434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:05.130899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea992bb-c5d5-4919-8821-4c456f899434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.132270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea992bb-c5d5-4919-8821-4c456f899434 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.144249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 255/511] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:05.145033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea992bb-c5d5-4919-8821-4c456f899434 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.161186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158feac8-b88d-4010-9576-5e4428eb67fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.161819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158feac8-b88d-4010-9576-5e4428eb67fa None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.162095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158feac8-b88d-4010-9576-5e4428eb67fa None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.162371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158feac8-b88d-4010-9576-5e4428eb67fa None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.163095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158feac8-b88d-4010-9576-5e4428eb67fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:05.163401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158feac8-b88d-4010-9576-5e4428eb67fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.164811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158feac8-b88d-4010-9576-5e4428eb67fa None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.200843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 257/512] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:05.201673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158feac8-b88d-4010-9576-5e4428eb67fa None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.243272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea992bb-c5d5-4919-8821-4c456f899434 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.277002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33494b27-f173-4f16-9160-5db351e62903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.277653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33494b27-f173-4f16-9160-5db351e62903 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.277925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33494b27-f173-4f16-9160-5db351e62903 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.278202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33494b27-f173-4f16-9160-5db351e62903 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.284704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33494b27-f173-4f16-9160-5db351e62903 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.286204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33494b27-f173-4f16-9160-5db351e62903 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.298825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158feac8-b88d-4010-9576-5e4428eb67fa None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-33494b27-f173-4f16-9160-5db351e62903 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:05.298976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:05.299906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:05.300714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:05.300714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:05.300714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:05.300714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:05.300714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:05.300714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:05.300714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:05.300714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:05.300714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 256/513] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:05.301181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33494b27-f173-4f16-9160-5db351e62903 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.318132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:05.334745 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf571c0-7ca2-476c-a45a-c99bc512d904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.335353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf571c0-7ca2-476c-a45a-c99bc512d904 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.335628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf571c0-7ca2-476c-a45a-c99bc512d904 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.335782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:05.335877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf571c0-7ca2-476c-a45a-c99bc512d904 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.336525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf571c0-7ca2-476c-a45a-c99bc512d904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:05.336863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf571c0-7ca2-476c-a45a-c99bc512d904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.338293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf571c0-7ca2-476c-a45a-c99bc512d904 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.349271 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 258/514] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:05.350174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf571c0-7ca2-476c-a45a-c99bc512d904 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.400312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33494b27-f173-4f16-9160-5db351e62903 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.432486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24cb2f-5388-49b5-9181-2336aa7f276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.433110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24cb2f-5388-49b5-9181-2336aa7f276e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.433394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24cb2f-5388-49b5-9181-2336aa7f276e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.433693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24cb2f-5388-49b5-9181-2336aa7f276e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.445168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24cb2f-5388-49b5-9181-2336aa7f276e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.446651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24cb2f-5388-49b5-9181-2336aa7f276e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.450515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf571c0-7ca2-476c-a45a-c99bc512d904 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-4e24cb2f-5388-49b5-9181-2336aa7f276e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:05.460958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:05.461833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:05.462637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:05.462637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:05.462637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:05.462637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 257/515] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:05.462858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24cb2f-5388-49b5-9181-2336aa7f276e None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.487480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd42213-9567-4ba4-9715-ecb6b79a64da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.488182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd42213-9567-4ba4-9715-ecb6b79a64da None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.488465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd42213-9567-4ba4-9715-ecb6b79a64da None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.488737 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd42213-9567-4ba4-9715-ecb6b79a64da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.496143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd42213-9567-4ba4-9715-ecb6b79a64da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.497559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd42213-9567-4ba4-9715-ecb6b79a64da None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-efd42213-9567-4ba4-9715-ecb6b79a64da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:05.510305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:05.511162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:05.512017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 259/516] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:05.512549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd42213-9567-4ba4-9715-ecb6b79a64da None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.551348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:05.553225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:05.569147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24cb2f-5388-49b5-9181-2336aa7f276e None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.600925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30c7c3-af3b-4b2c-9b53-681861b7cf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.601539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30c7c3-af3b-4b2c-9b53-681861b7cf61 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.601821 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30c7c3-af3b-4b2c-9b53-681861b7cf61 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.602090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30c7c3-af3b-4b2c-9b53-681861b7cf61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.612619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd42213-9567-4ba4-9715-ecb6b79a64da None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.645488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090f954-8f12-41bd-8d22-49dc2a7d1998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.646118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090f954-8f12-41bd-8d22-49dc2a7d1998 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.646393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090f954-8f12-41bd-8d22-49dc2a7d1998 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.646658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090f954-8f12-41bd-8d22-49dc2a7d1998 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.647332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090f954-8f12-41bd-8d22-49dc2a7d1998 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:05.647654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090f954-8f12-41bd-8d22-49dc2a7d1998 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.649078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090f954-8f12-41bd-8d22-49dc2a7d1998 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.666422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 260/517] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:05.667084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090f954-8f12-41bd-8d22-49dc2a7d1998 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.670411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30c7c3-af3b-4b2c-9b53-681861b7cf61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.672959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30c7c3-af3b-4b2c-9b53-681861b7cf61 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.754943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:05.761361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:05.765835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090f954-8f12-41bd-8d22-49dc2a7d1998 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.770501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 258/518] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:09:05.771312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30c7c3-af3b-4b2c-9b53-681861b7cf61 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.803263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2db0b1-9a9c-4ee4-8926-75231305750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.803863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2db0b1-9a9c-4ee4-8926-75231305750d None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.804142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2db0b1-9a9c-4ee4-8926-75231305750d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.804412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2db0b1-9a9c-4ee4-8926-75231305750d None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.805050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2db0b1-9a9c-4ee4-8926-75231305750d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.806496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2db0b1-9a9c-4ee4-8926-75231305750d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.827158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 261/519] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:09:05 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:05.827962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2db0b1-9a9c-4ee4-8926-75231305750d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.884133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30c7c3-af3b-4b2c-9b53-681861b7cf61 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.921690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b329eeb-3597-44d5-95cd-85713e42e8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.922326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b329eeb-3597-44d5-95cd-85713e42e8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.922606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b329eeb-3597-44d5-95cd-85713e42e8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.922880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b329eeb-3597-44d5-95cd-85713e42e8a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.924138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b329eeb-3597-44d5-95cd-85713e42e8a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:05.924442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b329eeb-3597-44d5-95cd-85713e42e8a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.925877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b329eeb-3597-44d5-95cd-85713e42e8a2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.942784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 259/520] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:05.942784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b329eeb-3597-44d5-95cd-85713e42e8a2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:05.949315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2db0b1-9a9c-4ee4-8926-75231305750d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:05.987616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2bd2b-1549-439c-8f24-fcf8dcbfe7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:05.988238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2bd2b-1549-439c-8f24-fcf8dcbfe7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:05.988511 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2bd2b-1549-439c-8f24-fcf8dcbfe7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:05.988783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2bd2b-1549-439c-8f24-fcf8dcbfe7f6 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:05.989555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2bd2b-1549-439c-8f24-fcf8dcbfe7f6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:05.989919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2bd2b-1549-439c-8f24-fcf8dcbfe7f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:05.991398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2bd2b-1549-439c-8f24-fcf8dcbfe7f6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:05.991946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:06.011363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 262/521] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:06.012011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2bd2b-1549-439c-8f24-fcf8dcbfe7f6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:06.029150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:06.060744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b329eeb-3597-44d5-95cd-85713e42e8a2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.095032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed70e3e-8bb7-45e6-be61-951202789c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:06.095664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed70e3e-8bb7-45e6-be61-951202789c7d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:06.095937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed70e3e-8bb7-45e6-be61-951202789c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:06.096202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed70e3e-8bb7-45e6-be61-951202789c7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:06.096948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed70e3e-8bb7-45e6-be61-951202789c7d None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=286f4e203e7a4a55bca0a58d061af2bd) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:06.097248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed70e3e-8bb7-45e6-be61-951202789c7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:06.098709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed70e3e-8bb7-45e6-be61-951202789c7d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:06.104297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 260/522] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:05 2022] GET /identity/v3/role_assignments?user.id=286f4e203e7a4a55bca0a58d061af2bd&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:06.105142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed70e3e-8bb7-45e6-be61-951202789c7d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:06.134419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2bd2b-1549-439c-8f24-fcf8dcbfe7f6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.176654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be387b35-797d-4c18-841b-2b2bbd891b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:06.177724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be387b35-797d-4c18-841b-2b2bbd891b17 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:06.178203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be387b35-797d-4c18-841b-2b2bbd891b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:06.178651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be387b35-797d-4c18-841b-2b2bbd891b17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:06.191764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be387b35-797d-4c18-841b-2b2bbd891b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:06.193861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be387b35-797d-4c18-841b-2b2bbd891b17 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-be387b35-797d-4c18-841b-2b2bbd891b17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:06.212101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:06.213042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:06.214028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:06.214028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:06.214028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:06.217953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 263/523] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:06.219135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be387b35-797d-4c18-841b-2b2bbd891b17 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:06.241159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed70e3e-8bb7-45e6-be61-951202789c7d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.254277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:06.281567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fe906-7866-4855-aabb-91a7ffd044b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:06.282506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fe906-7866-4855-aabb-91a7ffd044b5 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:06.283551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fe906-7866-4855-aabb-91a7ffd044b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:06.283995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fe906-7866-4855-aabb-91a7ffd044b5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:06.292757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fe906-7866-4855-aabb-91a7ffd044b5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:06.295302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fe906-7866-4855-aabb-91a7ffd044b5 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:06.295412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-c96fe906-7866-4855-aabb-91a7ffd044b5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:06.309187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:06.310128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 20:09:06.310992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:06.313481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 261/524] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:06.314693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fe906-7866-4855-aabb-91a7ffd044b5 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:06.337675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be387b35-797d-4c18-841b-2b2bbd891b17 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.384360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c5e42-77e9-48e3-b29d-b7431c491168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:06.385142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c5e42-77e9-48e3-b29d-b7431c491168 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:06.385487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c5e42-77e9-48e3-b29d-b7431c491168 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:06.385878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c5e42-77e9-48e3-b29d-b7431c491168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:06.424069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fe906-7866-4855-aabb-91a7ffd044b5 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.460413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ef64b-d9f8-4dd2-8b83-b7f1e06489c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:06.461310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ef64b-d9f8-4dd2-8b83-b7f1e06489c9 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:06.461793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ef64b-d9f8-4dd2-8b83-b7f1e06489c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:06.462264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ef64b-d9f8-4dd2-8b83-b7f1e06489c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:06.463108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ef64b-d9f8-4dd2-8b83-b7f1e06489c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:06.463633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ef64b-d9f8-4dd2-8b83-b7f1e06489c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:06.465180 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ef64b-d9f8-4dd2-8b83-b7f1e06489c9 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:06.470641 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c5e42-77e9-48e3-b29d-b7431c491168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:06.472120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c5e42-77e9-48e3-b29d-b7431c491168 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:06.478263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 262/525] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:06.480452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ef64b-d9f8-4dd2-8b83-b7f1e06489c9 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:06.515364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:06.545970 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:06.596345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ef64b-d9f8-4dd2-8b83-b7f1e06489c9 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.604218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 264/526] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:09:06.604916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c5e42-77e9-48e3-b29d-b7431c491168 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:06.640409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57615ce-913a-47d3-82bf-bce3552b4071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:06.641427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57615ce-913a-47d3-82bf-bce3552b4071 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:06.641952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57615ce-913a-47d3-82bf-bce3552b4071 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:06.643818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57615ce-913a-47d3-82bf-bce3552b4071 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:06.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57615ce-913a-47d3-82bf-bce3552b4071 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:06.645256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57615ce-913a-47d3-82bf-bce3552b4071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:06.647001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57615ce-913a-47d3-82bf-bce3552b4071 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:06.674907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 263/527] 10.222.0.23 () {56 vars in 1139 bytes} [Wed Oct 5 20:09:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:06.674907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57615ce-913a-47d3-82bf-bce3552b4071 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:06.734218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c5e42-77e9-48e3-b29d-b7431c491168 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.767877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:06.780035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67719c-1e2e-4e94-8d28-ac582cc65707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:06.780806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67719c-1e2e-4e94-8d28-ac582cc65707 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:06.781190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67719c-1e2e-4e94-8d28-ac582cc65707 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:06.781472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67719c-1e2e-4e94-8d28-ac582cc65707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:06.787513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57615ce-913a-47d3-82bf-bce3552b4071 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.789154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67719c-1e2e-4e94-8d28-ac582cc65707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:06.791354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67719c-1e2e-4e94-8d28-ac582cc65707 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-bc67719c-1e2e-4e94-8d28-ac582cc65707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:06.808965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:06.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:06.810720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:06.810720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:06.810720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:06.810720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:06.810720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:06.810720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:06.810720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:06.810720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:06.813574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 265/528] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:06.815654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67719c-1e2e-4e94-8d28-ac582cc65707 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:06.831425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aae60-d39c-4c38-920b-bec3a8ec8cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:06.832221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aae60-d39c-4c38-920b-bec3a8ec8cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:06.832636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aae60-d39c-4c38-920b-bec3a8ec8cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:06.833010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aae60-d39c-4c38-920b-bec3a8ec8cd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:06.843805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aae60-d39c-4c38-920b-bec3a8ec8cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:06.845729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aae60-d39c-4c38-920b-bec3a8ec8cd3 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:06.849489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-785aae60-d39c-4c38-920b-bec3a8ec8cd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:06.863334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:06.864378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:06.865182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:06.865182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:06.865182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:06.865182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 264/529] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:06.870569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aae60-d39c-4c38-920b-bec3a8ec8cd3 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:06.955253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67719c-1e2e-4e94-8d28-ac582cc65707 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.988664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aae60-d39c-4c38-920b-bec3a8ec8cd3 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:06.990417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1db01e-ff82-4cb0-8d95-bcfc258c8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:06.991310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1db01e-ff82-4cb0-8d95-bcfc258c8c8e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:06.991810 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1db01e-ff82-4cb0-8d95-bcfc258c8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:06.992283 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1db01e-ff82-4cb0-8d95-bcfc258c8c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:07.033630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6397839-b603-4e55-bbba-436f9446864f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:07.034244 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6397839-b603-4e55-bbba-436f9446864f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:07.034524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6397839-b603-4e55-bbba-436f9446864f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:07.034794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6397839-b603-4e55-bbba-436f9446864f None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:07.035494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6397839-b603-4e55-bbba-436f9446864f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:07.035786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6397839-b603-4e55-bbba-436f9446864f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:07.037239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6397839-b603-4e55-bbba-436f9446864f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:07.056700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:07.062360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 265/530] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:07.062360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6397839-b603-4e55-bbba-436f9446864f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:07.073030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1db01e-ff82-4cb0-8d95-bcfc258c8c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:07.074774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1db01e-ff82-4cb0-8d95-bcfc258c8c8e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:07.101722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:07.178791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6397839-b603-4e55-bbba-436f9446864f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:07.201682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 266/531] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:09:07.202545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1db01e-ff82-4cb0-8d95-bcfc258c8c8e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:07.218314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8865ec-b9d7-4467-a446-3a586972950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:07.218919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8865ec-b9d7-4467-a446-3a586972950a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:07.219199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8865ec-b9d7-4467-a446-3a586972950a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:07.219472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8865ec-b9d7-4467-a446-3a586972950a None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:07.220127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8865ec-b9d7-4467-a446-3a586972950a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:07.220942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8865ec-b9d7-4467-a446-3a586972950a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:07.222346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8865ec-b9d7-4467-a446-3a586972950a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:07.234690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 266/532] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:07.235516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8865ec-b9d7-4467-a446-3a586972950a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:07.320562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:07.327733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1db01e-ff82-4cb0-8d95-bcfc258c8c8e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:07.361146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:07.367578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8865ec-b9d7-4467-a446-3a586972950a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:07.375779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd4d57-123c-4639-bfb8-ca82a25255b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:07.377195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd4d57-123c-4639-bfb8-ca82a25255b0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:07.377726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd4d57-123c-4639-bfb8-ca82a25255b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:07.378638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd4d57-123c-4639-bfb8-ca82a25255b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:07.379967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd4d57-123c-4639-bfb8-ca82a25255b0 None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=47bafea2367e4c7992354bb2c23e7d57) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:07.380749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd4d57-123c-4639-bfb8-ca82a25255b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:07.382685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd4d57-123c-4639-bfb8-ca82a25255b0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:07.395671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 267/533] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:07 2022] GET /identity/v3/role_assignments?user.id=47bafea2367e4c7992354bb2c23e7d57&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:07.395671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd4d57-123c-4639-bfb8-ca82a25255b0 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:07.422630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dfe17-356c-48e2-9cdf-deb8a56122cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:07.423393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dfe17-356c-48e2-9cdf-deb8a56122cf None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:07.423673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dfe17-356c-48e2-9cdf-deb8a56122cf None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:07.423945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dfe17-356c-48e2-9cdf-deb8a56122cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:07.444283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dfe17-356c-48e2-9cdf-deb8a56122cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:07.445809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dfe17-356c-48e2-9cdf-deb8a56122cf None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-697dfe17-356c-48e2-9cdf-deb8a56122cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:07.473132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:07.474100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:07.474893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:07.474893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:07.474893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:07.482194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 267/534] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:07.486291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dfe17-356c-48e2-9cdf-deb8a56122cf None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:07.597133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd4d57-123c-4639-bfb8-ca82a25255b0 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:07.652578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c698f7-3984-4047-8f9a-d7fa149ca3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:07.653599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c698f7-3984-4047-8f9a-d7fa149ca3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:07.654068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c698f7-3984-4047-8f9a-d7fa149ca3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:07.654523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c698f7-3984-4047-8f9a-d7fa149ca3b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:07.663121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c698f7-3984-4047-8f9a-d7fa149ca3b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:07.665467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c698f7-3984-4047-8f9a-d7fa149ca3b3 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-04c698f7-3984-4047-8f9a-d7fa149ca3b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:07.679288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:07.680109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:07.680904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:07.680904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:07.680904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:07.680904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:07.680904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:07.680904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:07.680904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:07.680904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:07.687766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dfe17-356c-48e2-9cdf-deb8a56122cf None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:07.688093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 268/535] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:07.689817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c698f7-3984-4047-8f9a-d7fa149ca3b3 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:07.714522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:07.743832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c73aa9-a7ec-422d-b72a-b52dcd91f2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:07.744440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c73aa9-a7ec-422d-b72a-b52dcd91f2ab None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:07.744717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c73aa9-a7ec-422d-b72a-b52dcd91f2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:07.744988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c73aa9-a7ec-422d-b72a-b52dcd91f2ab None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:07.745660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c73aa9-a7ec-422d-b72a-b52dcd91f2ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:07.746732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c73aa9-a7ec-422d-b72a-b52dcd91f2ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:07.749541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c73aa9-a7ec-422d-b72a-b52dcd91f2ab None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:07.775661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 268/536] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:07.781275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c73aa9-a7ec-422d-b72a-b52dcd91f2ab None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:07.789868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:07.864742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c698f7-3984-4047-8f9a-d7fa149ca3b3 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:07.900490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16be244-536a-449c-af2e-6598aae84131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:07.901097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16be244-536a-449c-af2e-6598aae84131 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:07.901375 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16be244-536a-449c-af2e-6598aae84131 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:07.901687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16be244-536a-449c-af2e-6598aae84131 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:07.902377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16be244-536a-449c-af2e-6598aae84131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:07.902678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16be244-536a-449c-af2e-6598aae84131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:07.904097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16be244-536a-449c-af2e-6598aae84131 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:07.917723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 269/537] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:07.923554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16be244-536a-449c-af2e-6598aae84131 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:07.997100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c73aa9-a7ec-422d-b72a-b52dcd91f2ab None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:08.040849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1653d-4dbd-4e03-839a-f8140411d36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:08.041485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1653d-4dbd-4e03-839a-f8140411d36f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:08.041832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1653d-4dbd-4e03-839a-f8140411d36f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:08.042107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1653d-4dbd-4e03-839a-f8140411d36f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:08.049502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1653d-4dbd-4e03-839a-f8140411d36f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:08.051239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1653d-4dbd-4e03-839a-f8140411d36f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-c4b1653d-4dbd-4e03-839a-f8140411d36f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:08.063640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:08.065067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:08.066686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:08.066686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:08.066686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:08.066686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:08.066686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:08.066686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:08.066686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:08.066686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:08.066686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 269/538] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:08.071417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1653d-4dbd-4e03-839a-f8140411d36f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:08.109915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:08.164889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16be244-536a-449c-af2e-6598aae84131 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:08.188671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1653d-4dbd-4e03-839a-f8140411d36f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:08.193906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:08.203275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9ed25-c016-4914-8c12-60df933baf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:08.203954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9ed25-c016-4914-8c12-60df933baf83 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:08.204169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9ed25-c016-4914-8c12-60df933baf83 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:08.204481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9ed25-c016-4914-8c12-60df933baf83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:08.206089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9ed25-c016-4914-8c12-60df933baf83 None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=a6e8bc66986d48aa9dcbc19258ff6e1f) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:08.206404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9ed25-c016-4914-8c12-60df933baf83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:08.208028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9ed25-c016-4914-8c12-60df933baf83 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:08.216005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 270/539] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:07 2022] GET /identity/v3/role_assignments?user.id=a6e8bc66986d48aa9dcbc19258ff6e1f&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:08.221865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbff00-f3ba-4055-abd7-76322278b6b0 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:08.221865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbff00-f3ba-4055-abd7-76322278b6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:08.221865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbff00-f3ba-4055-abd7-76322278b6b0 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:08.221865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 271/540] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:08.239411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493729-f608-498e-ab84-3592ff5bd9dd None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:08.239762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493729-f608-498e-ab84-3592ff5bd9dd None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:08.240008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45493729-f608-498e-ab84-3592ff5bd9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:08.248898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ffcd4-b5cc-4423-8d68-66fb88354da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:08.249633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ffcd4-b5cc-4423-8d68-66fb88354da5 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:08.249915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ffcd4-b5cc-4423-8d68-66fb88354da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:08.250187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ffcd4-b5cc-4423-8d68-66fb88354da5 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:08.252992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ffcd4-b5cc-4423-8d68-66fb88354da5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:08.253305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ffcd4-b5cc-4423-8d68-66fb88354da5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:08.258224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ffcd4-b5cc-4423-8d68-66fb88354da5 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:08.271492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-45493729-f608-498e-ab84-3592ff5bd9dd None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:08.272753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-45493729-f608-498e-ab84-3592ff5bd9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:08.274527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 270/541] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:08.280864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ffcd4-b5cc-4423-8d68-66fb88354da5 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:08.398863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:08.402917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ffcd4-b5cc-4423-8d68-66fb88354da5 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:08.438407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17d977-e56d-48b7-9a91-f0c684fbd752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:08.438672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 272/542] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:09:08.440027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17d977-e56d-48b7-9a91-f0c684fbd752 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:08.440398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17d977-e56d-48b7-9a91-f0c684fbd752 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:08.440766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17d977-e56d-48b7-9a91-f0c684fbd752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:08.440879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79dfab-93f4-4aa1-9a45-cbd06e2d0c80 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:08.441050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79dfab-93f4-4aa1-9a45-cbd06e2d0c80 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:08.441319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79dfab-93f4-4aa1-9a45-cbd06e2d0c80 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:08.441911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17d977-e56d-48b7-9a91-f0c684fbd752 None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=8f5a88f263014f15a6f59cfdaa2bb6d8) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:08.442314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17d977-e56d-48b7-9a91-f0c684fbd752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:08.443930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f17d977-e56d-48b7-9a91-f0c684fbd752 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:08.446793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 273/543] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:08.447960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b2ae6-4716-46ab-a1df-380342bc9095 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:08.448247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b2ae6-4716-46ab-a1df-380342bc9095 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:08.448523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b2ae6-4716-46ab-a1df-380342bc9095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:08.450480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 271/544] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:08 2022] GET /identity/v3/role_assignments?user.id=8f5a88f263014f15a6f59cfdaa2bb6d8&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:08.471962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-809b2ae6-4716-46ab-a1df-380342bc9095 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:08.473363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-809b2ae6-4716-46ab-a1df-380342bc9095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:08.581330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 274/545] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:09:08.740157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17d977-e56d-48b7-9a91-f0c684fbd752 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:08.849429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17d977-e56d-48b7-9a91-f0c684fbd752 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:08.896176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01be812-2992-4d70-a793-42133847435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:08.896813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01be812-2992-4d70-a793-42133847435d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:08.897089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01be812-2992-4d70-a793-42133847435d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:08.897360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01be812-2992-4d70-a793-42133847435d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:08.914481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01be812-2992-4d70-a793-42133847435d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-c01be812-2992-4d70-a793-42133847435d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:08.919716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:08.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:08.921719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 272/546] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:08.928943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b2ae6-4716-46ab-a1df-380342bc9095 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:08.962210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:08.987166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01be812-2992-4d70-a793-42133847435d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:09.019102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:09.059482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b2ae6-4716-46ab-a1df-380342bc9095 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:09.120463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc128695-9e7c-4a0d-86c0-d06b6451fa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:09.121066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc128695-9e7c-4a0d-86c0-d06b6451fa44 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:09.121341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc128695-9e7c-4a0d-86c0-d06b6451fa44 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:09.121630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc128695-9e7c-4a0d-86c0-d06b6451fa44 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:09.122290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc128695-9e7c-4a0d-86c0-d06b6451fa44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:09.122626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc128695-9e7c-4a0d-86c0-d06b6451fa44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:09.124001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc128695-9e7c-4a0d-86c0-d06b6451fa44 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:09.132793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 275/547] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:09.138541 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc128695-9e7c-4a0d-86c0-d06b6451fa44 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:09.175028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01be812-2992-4d70-a793-42133847435d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:09.225828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5f186-7231-40cf-9c97-2b9dd7d03f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:09.226448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5f186-7231-40cf-9c97-2b9dd7d03f1f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:09.226723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5f186-7231-40cf-9c97-2b9dd7d03f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:09.226971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5f186-7231-40cf-9c97-2b9dd7d03f1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:09.240352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5f186-7231-40cf-9c97-2b9dd7d03f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-a7d5f186-7231-40cf-9c97-2b9dd7d03f1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:09.245483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:09.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:09.247531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 273/548] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:09.252503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5f186-7231-40cf-9c97-2b9dd7d03f1f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:09.312338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:09.321560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc128695-9e7c-4a0d-86c0-d06b6451fa44 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:09.361410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b7c7e-1c70-4e6f-96d5-8cab0ccd14e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:09.361410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b7c7e-1c70-4e6f-96d5-8cab0ccd14e4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:09.361410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b7c7e-1c70-4e6f-96d5-8cab0ccd14e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:09.361410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b7c7e-1c70-4e6f-96d5-8cab0ccd14e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:09.381633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:09.386840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b7c7e-1c70-4e6f-96d5-8cab0ccd14e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:09.388354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b7c7e-1c70-4e6f-96d5-8cab0ccd14e4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-983b7c7e-1c70-4e6f-96d5-8cab0ccd14e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:09.410627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:09.412334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:09.413336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:09.413336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:09.413336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:09.413336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 276/549] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:09.419340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b7c7e-1c70-4e6f-96d5-8cab0ccd14e4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:09.421637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5f186-7231-40cf-9c97-2b9dd7d03f1f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:09.480714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc2b6a-c91b-49bb-b002-e87056762559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:09.482766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc2b6a-c91b-49bb-b002-e87056762559 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:09.483266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc2b6a-c91b-49bb-b002-e87056762559 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:09.483727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc2b6a-c91b-49bb-b002-e87056762559 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:09.485346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc2b6a-c91b-49bb-b002-e87056762559 None admin] RBAC: Adding query filter params (name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:09.487718 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc2b6a-c91b-49bb-b002-e87056762559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:09.489371 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc2b6a-c91b-49bb-b002-e87056762559 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:09.507156 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 274/550] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:09.512917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc2b6a-c91b-49bb-b002-e87056762559 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:09.602022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b7c7e-1c70-4e6f-96d5-8cab0ccd14e4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:09.655986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7fd89-ae91-4959-9d20-b13fe4faea6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:09.656937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7fd89-ae91-4959-9d20-b13fe4faea6b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:09.657430 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7fd89-ae91-4959-9d20-b13fe4faea6b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:09.657935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7fd89-ae91-4959-9d20-b13fe4faea6b None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:09.658786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7fd89-ae91-4959-9d20-b13fe4faea6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:09.659341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7fd89-ae91-4959-9d20-b13fe4faea6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:09.660946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7fd89-ae91-4959-9d20-b13fe4faea6b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:09.677629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 277/551] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:09.679218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7fd89-ae91-4959-9d20-b13fe4faea6b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:09.690586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:09.724068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc2b6a-c91b-49bb-b002-e87056762559 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:09.786841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8142f-9545-4988-8e44-2127a124fc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:09.787850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8142f-9545-4988-8e44-2127a124fc0b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:09.788332 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8142f-9545-4988-8e44-2127a124fc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:09.788792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8142f-9545-4988-8e44-2127a124fc0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:09.796646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:09.804047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8142f-9545-4988-8e44-2127a124fc0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:09.805805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8142f-9545-4988-8e44-2127a124fc0b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-b9b8142f-9545-4988-8e44-2127a124fc0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:09.832857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:09.834262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:09.835093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:09.835093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:09.835093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:09.841149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 275/552] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:09.847087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8142f-9545-4988-8e44-2127a124fc0b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:09.855449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7fd89-ae91-4959-9d20-b13fe4faea6b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:09.925280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa367660-51e7-4a0d-9b0f-eeaa0e98ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:09.925966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa367660-51e7-4a0d-9b0f-eeaa0e98ae3e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:09.926206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa367660-51e7-4a0d-9b0f-eeaa0e98ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:09.926473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa367660-51e7-4a0d-9b0f-eeaa0e98ae3e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:09.938600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa367660-51e7-4a0d-9b0f-eeaa0e98ae3e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:09.942579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa367660-51e7-4a0d-9b0f-eeaa0e98ae3e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-aa367660-51e7-4a0d-9b0f-eeaa0e98ae3e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:09.964192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:09.965134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:09.966118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:09.973604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 278/553] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:09.975101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f9496-d4c8-452e-b666-ace31eacae3e None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:09.975572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f9496-d4c8-452e-b666-ace31eacae3e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:09.975980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f9496-d4c8-452e-b666-ace31eacae3e None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:09.977734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 279/554] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:09.983423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f9496-d4c8-452e-b666-ace31eacae3e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:10.025370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8142f-9545-4988-8e44-2127a124fc0b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:10.068414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaeb785-2f63-4e9d-94ac-47b33653d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:10.069200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb785-2f63-4e9d-94ac-47b33653d613 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.069619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb785-2f63-4e9d-94ac-47b33653d613 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.070362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:10.070634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaeb785-2f63-4e9d-94ac-47b33653d613 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:10.072548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeb785-2f63-4e9d-94ac-47b33653d613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:10.072953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeb785-2f63-4e9d-94ac-47b33653d613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:10.074984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaeb785-2f63-4e9d-94ac-47b33653d613 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:10.091604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 276/555] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:10.093348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836f2ff-1229-4e9a-834b-5911b592d284 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.093755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836f2ff-1229-4e9a-834b-5911b592d284 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.094169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836f2ff-1229-4e9a-834b-5911b592d284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:10.114989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f9496-d4c8-452e-b666-ace31eacae3e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:10.116416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e836f2ff-1229-4e9a-834b-5911b592d284 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:10.118819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e836f2ff-1229-4e9a-834b-5911b592d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:10.158507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a4591-afdb-4bd0-8106-b0a2985db458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:10.159361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a4591-afdb-4bd0-8106-b0a2985db458 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.159783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a4591-afdb-4bd0-8106-b0a2985db458 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.160260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a4591-afdb-4bd0-8106-b0a2985db458 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:10.161327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a4591-afdb-4bd0-8106-b0a2985db458 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:10.161796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a4591-afdb-4bd0-8106-b0a2985db458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:10.163708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a4591-afdb-4bd0-8106-b0a2985db458 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:10.179242 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:10.186947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 280/556] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 20:09:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:10.190850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a4591-afdb-4bd0-8106-b0a2985db458 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:10.209262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 277/557] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:10.210474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836f2ff-1229-4e9a-834b-5911b592d284 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:10.310487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a4591-afdb-4bd0-8106-b0a2985db458 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:10.321101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836f2ff-1229-4e9a-834b-5911b592d284 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:10.339863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:10.364414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de6a26-4a1f-4b72-a249-f8f531219291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:10.365943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6a26-4a1f-4b72-a249-f8f531219291 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.367798 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6a26-4a1f-4b72-a249-f8f531219291 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.368268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de6a26-4a1f-4b72-a249-f8f531219291 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:10.372358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c715462-c8bd-492c-b08c-cd4a8fbd0408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:10.373229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c715462-c8bd-492c-b08c-cd4a8fbd0408 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.373926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c715462-c8bd-492c-b08c-cd4a8fbd0408 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.375222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c715462-c8bd-492c-b08c-cd4a8fbd0408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:10.386650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de6a26-4a1f-4b72-a249-f8f531219291 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:10.394495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de6a26-4a1f-4b72-a249-f8f531219291 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-69de6a26-4a1f-4b72-a249-f8f531219291 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:10.411955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:10.412940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:10.413849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:10.416417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 281/558] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 20:09:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:10.421488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de6a26-4a1f-4b72-a249-f8f531219291 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:10.469065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:10.508458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c715462-c8bd-492c-b08c-cd4a8fbd0408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:10.511328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c715462-c8bd-492c-b08c-cd4a8fbd0408 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:10.590340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de6a26-4a1f-4b72-a249-f8f531219291 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:10.637018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a44ee-2e0c-4dbd-9d5d-05627d2a5ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:10.638231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a44ee-2e0c-4dbd-9d5d-05627d2a5ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.638519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a44ee-2e0c-4dbd-9d5d-05627d2a5ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.638786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a44ee-2e0c-4dbd-9d5d-05627d2a5ec7 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:10.639515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a44ee-2e0c-4dbd-9d5d-05627d2a5ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:10.639815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a44ee-2e0c-4dbd-9d5d-05627d2a5ec7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:10.641245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a44ee-2e0c-4dbd-9d5d-05627d2a5ec7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:10.665190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 278/559] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:09:10.665496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 282/560] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 20:09:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:10.666066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c715462-c8bd-492c-b08c-cd4a8fbd0408 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:10.672008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a44ee-2e0c-4dbd-9d5d-05627d2a5ec7 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:10.719542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:10.761294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:10.783611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c715462-c8bd-492c-b08c-cd4a8fbd0408 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:10.785462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a44ee-2e0c-4dbd-9d5d-05627d2a5ec7 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:10.821409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af9e45-4236-4c8f-b1ff-9aae66716846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:10.822131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f7c8c-0c3a-4592-a698-e7aeef94f7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:10.822678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af9e45-4236-4c8f-b1ff-9aae66716846 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.822786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f7c8c-0c3a-4592-a698-e7aeef94f7da None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.823017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f7c8c-0c3a-4592-a698-e7aeef94f7da None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.823280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f7c8c-0c3a-4592-a698-e7aeef94f7da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:10.823669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af9e45-4236-4c8f-b1ff-9aae66716846 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.824128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af9e45-4236-4c8f-b1ff-9aae66716846 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:10.834831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f7c8c-0c3a-4592-a698-e7aeef94f7da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:10.835834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af9e45-4236-4c8f-b1ff-9aae66716846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:10.836305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f7c8c-0c3a-4592-a698-e7aeef94f7da None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-66af9e45-4236-4c8f-b1ff-9aae66716846 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:10.840803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:10.842059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:10.842865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 279/561] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:10.843595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af9e45-4236-4c8f-b1ff-9aae66716846 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-a43f7c8c-0c3a-4592-a698-e7aeef94f7da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:10.853669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:10.854657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:10.856140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:10.856140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:10.856140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:10.856140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 283/562] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:10.859056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600acbd6-83ca-42e3-be27-7e41f440ff3d None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.859763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600acbd6-83ca-42e3-be27-7e41f440ff3d None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.860112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600acbd6-83ca-42e3-be27-7e41f440ff3d None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:10.864586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 284/563] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:10.864586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600acbd6-83ca-42e3-be27-7e41f440ff3d None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:10.960220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af9e45-4236-4c8f-b1ff-9aae66716846 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:10.969452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:10.977995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600acbd6-83ca-42e3-be27-7e41f440ff3d None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:10.994428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9c3ab-9985-40b1-ba5e-506d693c2cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:10.995045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c3ab-9985-40b1-ba5e-506d693c2cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:10.995324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c3ab-9985-40b1-ba5e-506d693c2cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:10.995594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9c3ab-9985-40b1-ba5e-506d693c2cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.003606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:11.006023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9c3ab-9985-40b1-ba5e-506d693c2cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.007693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9c3ab-9985-40b1-ba5e-506d693c2cc6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.012450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83cf8f-58f2-4894-93c3-4bd688b3f15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.013041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83cf8f-58f2-4894-93c3-4bd688b3f15e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.013314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83cf8f-58f2-4894-93c3-4bd688b3f15e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.013614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83cf8f-58f2-4894-93c3-4bd688b3f15e None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.014528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83cf8f-58f2-4894-93c3-4bd688b3f15e None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:11.014826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83cf8f-58f2-4894-93c3-4bd688b3f15e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.016171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83cf8f-58f2-4894-93c3-4bd688b3f15e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-96f9c3ab-9985-40b1-ba5e-506d693c2cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:11.022202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:11.023133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:11.023989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:11.023989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:11.023989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:11.023989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 280/564] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:11.023989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9c3ab-9985-40b1-ba5e-506d693c2cc6 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.024706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 285/565] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:11.025788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461b132-a499-4c09-9954-82d119e39493 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.026066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461b132-a499-4c09-9954-82d119e39493 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.026338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461b132-a499-4c09-9954-82d119e39493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.043673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-5461b132-a499-4c09-9954-82d119e39493 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:11.044849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5461b132-a499-4c09-9954-82d119e39493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:11.108525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 286/566] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:11.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a26f97-81e1-4bc1-af1d-43ab307a8e34 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.110027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a26f97-81e1-4bc1-af1d-43ab307a8e34 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.110297 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a26f97-81e1-4bc1-af1d-43ab307a8e34 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.113564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 287/567] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:11.113564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a26f97-81e1-4bc1-af1d-43ab307a8e34 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.127732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9c3ab-9985-40b1-ba5e-506d693c2cc6 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.161030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6ee37-b464-466f-9805-015cb015f68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.161650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6ee37-b464-466f-9805-015cb015f68b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.161925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6ee37-b464-466f-9805-015cb015f68b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.162194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6ee37-b464-466f-9805-015cb015f68b None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.162823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6ee37-b464-466f-9805-015cb015f68b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:11.163118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6ee37-b464-466f-9805-015cb015f68b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.164477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6ee37-b464-466f-9805-015cb015f68b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.177545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 281/568] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:11.177850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6ee37-b464-466f-9805-015cb015f68b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.218529 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:11.227629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:11.238927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a26f97-81e1-4bc1-af1d-43ab307a8e34 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.273335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6ee37-b464-466f-9805-015cb015f68b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.276754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3eb29-6b31-4ff3-96b9-835f99ae398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.277354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3eb29-6b31-4ff3-96b9-835f99ae398c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.277658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3eb29-6b31-4ff3-96b9-835f99ae398c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.277926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3eb29-6b31-4ff3-96b9-835f99ae398c None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.278555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3eb29-6b31-4ff3-96b9-835f99ae398c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:11.278853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3eb29-6b31-4ff3-96b9-835f99ae398c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.280203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3eb29-6b31-4ff3-96b9-835f99ae398c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.296494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 288/569] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:11.296494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fadb26-e3cf-417b-8d25-36b8d98b371c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.296494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fadb26-e3cf-417b-8d25-36b8d98b371c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.296494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fadb26-e3cf-417b-8d25-36b8d98b371c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.312218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e5fadb26-e3cf-417b-8d25-36b8d98b371c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:11.313379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e5fadb26-e3cf-417b-8d25-36b8d98b371c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:11.315856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce570eb-f2f2-48d1-b9ed-105370041712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.317272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce570eb-f2f2-48d1-b9ed-105370041712 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.317596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce570eb-f2f2-48d1-b9ed-105370041712 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.317869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce570eb-f2f2-48d1-b9ed-105370041712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.330348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce570eb-f2f2-48d1-b9ed-105370041712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.332074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce570eb-f2f2-48d1-b9ed-105370041712 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-2ce570eb-f2f2-48d1-b9ed-105370041712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:11.350072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:11.350970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:11.351846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:11.351846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:11.351846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:11.351846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 282/570] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:11.352498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce570eb-f2f2-48d1-b9ed-105370041712 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.387118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 289/571] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:11.388498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fadb26-e3cf-417b-8d25-36b8d98b371c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.456716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce570eb-f2f2-48d1-b9ed-105370041712 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.460380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:11.466654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:11.491309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fadb26-e3cf-417b-8d25-36b8d98b371c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.492975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1b9a8-4d00-4f79-affa-054fc8c1a81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.493629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1b9a8-4d00-4f79-affa-054fc8c1a81a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.493900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1b9a8-4d00-4f79-affa-054fc8c1a81a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.494175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1b9a8-4d00-4f79-affa-054fc8c1a81a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.501036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1b9a8-4d00-4f79-affa-054fc8c1a81a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.503117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1b9a8-4d00-4f79-affa-054fc8c1a81a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-eba1b9a8-4d00-4f79-affa-054fc8c1a81a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:11.513974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:11.514791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:11.515661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:11.515661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:11.515661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:11.515661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:11.515661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:11.515661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:11.515661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:11.515661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:11.519936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 283/572] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:11.520749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1b9a8-4d00-4f79-affa-054fc8c1a81a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.529842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454eed83-c5d9-44b4-9d66-7a0b115b4cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.530503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454eed83-c5d9-44b4-9d66-7a0b115b4cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.530779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454eed83-c5d9-44b4-9d66-7a0b115b4cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.531046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454eed83-c5d9-44b4-9d66-7a0b115b4cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.537985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454eed83-c5d9-44b4-9d66-7a0b115b4cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.539411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454eed83-c5d9-44b4-9d66-7a0b115b4cf8 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-454eed83-c5d9-44b4-9d66-7a0b115b4cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:11.550918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:11.551942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 290/573] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:11.557647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454eed83-c5d9-44b4-9d66-7a0b115b4cf8 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.631747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1b9a8-4d00-4f79-affa-054fc8c1a81a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.666261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4d34c-446a-452a-bfa6-e3240cfdfca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.666857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4d34c-446a-452a-bfa6-e3240cfdfca9 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.667129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4d34c-446a-452a-bfa6-e3240cfdfca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.667395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4d34c-446a-452a-bfa6-e3240cfdfca9 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.667776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454eed83-c5d9-44b4-9d66-7a0b115b4cf8 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.668024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4d34c-446a-452a-bfa6-e3240cfdfca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:11.668323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4d34c-446a-452a-bfa6-e3240cfdfca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.669723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4d34c-446a-452a-bfa6-e3240cfdfca9 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.683171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 284/574] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:11.683777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4d34c-446a-452a-bfa6-e3240cfdfca9 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.704939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41248752-18d2-42b3-a1fb-60e9a033ff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.705596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41248752-18d2-42b3-a1fb-60e9a033ff25 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.705874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41248752-18d2-42b3-a1fb-60e9a033ff25 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.706143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41248752-18d2-42b3-a1fb-60e9a033ff25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.707748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:11.709171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:11.717949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41248752-18d2-42b3-a1fb-60e9a033ff25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-41248752-18d2-42b3-a1fb-60e9a033ff25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:11.723316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:11.725560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 291/575] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:11.726643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41248752-18d2-42b3-a1fb-60e9a033ff25 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.784963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4d34c-446a-452a-bfa6-e3240cfdfca9 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.817798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d31d84-b860-422e-bb8d-878eb8965676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.818387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d31d84-b860-422e-bb8d-878eb8965676 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.818660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d31d84-b860-422e-bb8d-878eb8965676 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.818926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d31d84-b860-422e-bb8d-878eb8965676 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.819594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d31d84-b860-422e-bb8d-878eb8965676 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:11.819896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d31d84-b860-422e-bb8d-878eb8965676 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.821303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d31d84-b860-422e-bb8d-878eb8965676 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.822435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41248752-18d2-42b3-a1fb-60e9a033ff25 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.832771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 285/576] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:11.833567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d31d84-b860-422e-bb8d-878eb8965676 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.855844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901160a7-fe79-4c67-a756-6346e83c6b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.856441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901160a7-fe79-4c67-a756-6346e83c6b71 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.856719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901160a7-fe79-4c67-a756-6346e83c6b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.856989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901160a7-fe79-4c67-a756-6346e83c6b71 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.857709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901160a7-fe79-4c67-a756-6346e83c6b71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:11.858011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901160a7-fe79-4c67-a756-6346e83c6b71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:11.859418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901160a7-fe79-4c67-a756-6346e83c6b71 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:11.871020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 292/577] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:11.877015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901160a7-fe79-4c67-a756-6346e83c6b71 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:11.917578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:11.931031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:11.933906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d31d84-b860-422e-bb8d-878eb8965676 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.967070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b24886-9a5d-4c56-9021-2507a2044baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:11.967685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b24886-9a5d-4c56-9021-2507a2044baf None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:11.967965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b24886-9a5d-4c56-9021-2507a2044baf None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:11.968232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b24886-9a5d-4c56-9021-2507a2044baf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:11.974367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901160a7-fe79-4c67-a756-6346e83c6b71 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:11.977986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b24886-9a5d-4c56-9021-2507a2044baf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-82b24886-9a5d-4c56-9021-2507a2044baf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:11.983105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:11.984174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:11.984989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 286/578] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:11.985700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b24886-9a5d-4c56-9021-2507a2044baf None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.010755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f4d93-9b56-455b-9b19-9585aa368083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.011482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f4d93-9b56-455b-9b19-9585aa368083 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.011655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f4d93-9b56-455b-9b19-9585aa368083 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.011926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f4d93-9b56-455b-9b19-9585aa368083 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.019769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f4d93-9b56-455b-9b19-9585aa368083 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.021198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f4d93-9b56-455b-9b19-9585aa368083 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-751f4d93-9b56-455b-9b19-9585aa368083 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:12.031976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:12.032942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 293/579] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:12.034228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f4d93-9b56-455b-9b19-9585aa368083 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.087831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b24886-9a5d-4c56-9021-2507a2044baf None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:12.120971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c31e71-bdb7-4284-87c8-c834835cd91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.121595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c31e71-bdb7-4284-87c8-c834835cd91e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.121871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c31e71-bdb7-4284-87c8-c834835cd91e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.122139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c31e71-bdb7-4284-87c8-c834835cd91e None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.122784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c31e71-bdb7-4284-87c8-c834835cd91e None admin] RBAC: Adding query filter params (name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:12.123079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c31e71-bdb7-4284-87c8-c834835cd91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.124432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c31e71-bdb7-4284-87c8-c834835cd91e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.132397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 287/580] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:12.133160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c31e71-bdb7-4284-87c8-c834835cd91e None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.134734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f4d93-9b56-455b-9b19-9585aa368083 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:12.143582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:12.151612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:12.169740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18094bf6-f863-4184-aac4-acf87ecf242f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.170388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18094bf6-f863-4184-aac4-acf87ecf242f None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.170663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18094bf6-f863-4184-aac4-acf87ecf242f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.170933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18094bf6-f863-4184-aac4-acf87ecf242f None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/286f4e203e7a4a55bca0a58d061af2bd/roles/89ae1151daf54ba9b292e812aac0ecc6` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.184717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18094bf6-f863-4184-aac4-acf87ecf242f None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=286f4e203e7a4a55bca0a58d061af2bd, role_id=89ae1151daf54ba9b292e812aac0ecc6)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.186180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18094bf6-f863-4184-aac4-acf87ecf242f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.209544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 294/581] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:09:12 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/286f4e203e7a4a55bca0a58d061af2bd/roles/89ae1151daf54ba9b292e812aac0ecc6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:09:12.209544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18094bf6-f863-4184-aac4-acf87ecf242f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.240314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c31e71-bdb7-4284-87c8-c834835cd91e None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:12.280136 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e65fe-3908-4dc6-bdfd-b7f0a79cae22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.280751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e65fe-3908-4dc6-bdfd-b7f0a79cae22 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.281025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e65fe-3908-4dc6-bdfd-b7f0a79cae22 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.281292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e65fe-3908-4dc6-bdfd-b7f0a79cae22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.282080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e65fe-3908-4dc6-bdfd-b7f0a79cae22 None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=f8dec1dbdcd94316aad25c93c313a5a5) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:12.282381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e65fe-3908-4dc6-bdfd-b7f0a79cae22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.284414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e65fe-3908-4dc6-bdfd-b7f0a79cae22 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.290420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 288/582] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:12 2022] GET /identity/v3/role_assignments?user.id=f8dec1dbdcd94316aad25c93c313a5a5&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:12.291280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e65fe-3908-4dc6-bdfd-b7f0a79cae22 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.344336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18094bf6-f863-4184-aac4-acf87ecf242f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:12.393077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:12.396420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9917f9-4418-4b6a-bbae-2accbbc471d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.397242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9917f9-4418-4b6a-bbae-2accbbc471d7 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.397530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9917f9-4418-4b6a-bbae-2accbbc471d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.397801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9917f9-4418-4b6a-bbae-2accbbc471d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.398477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9917f9-4418-4b6a-bbae-2accbbc471d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:12.398777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9917f9-4418-4b6a-bbae-2accbbc471d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.400158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9917f9-4418-4b6a-bbae-2accbbc471d7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.412571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 295/583] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:12.413763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9917f9-4418-4b6a-bbae-2accbbc471d7 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.435069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e65fe-3908-4dc6-bdfd-b7f0a79cae22 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:12.456292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:12.474765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484cf06-1b50-45f0-86f3-3a5a62c7b36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.475384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484cf06-1b50-45f0-86f3-3a5a62c7b36d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.475682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484cf06-1b50-45f0-86f3-3a5a62c7b36d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.476026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484cf06-1b50-45f0-86f3-3a5a62c7b36d None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.476764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484cf06-1b50-45f0-86f3-3a5a62c7b36d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:12.477067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484cf06-1b50-45f0-86f3-3a5a62c7b36d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.478661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484cf06-1b50-45f0-86f3-3a5a62c7b36d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.499705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 289/584] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 20:09:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:12.500560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484cf06-1b50-45f0-86f3-3a5a62c7b36d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.555967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9917f9-4418-4b6a-bbae-2accbbc471d7 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:12.596693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8232ef0-e21d-45a9-83a8-48603c919b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.597242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8232ef0-e21d-45a9-83a8-48603c919b11 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.597544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8232ef0-e21d-45a9-83a8-48603c919b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.597829 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8232ef0-e21d-45a9-83a8-48603c919b11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.610591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484cf06-1b50-45f0-86f3-3a5a62c7b36d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:12.611655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8232ef0-e21d-45a9-83a8-48603c919b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.613206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8232ef0-e21d-45a9-83a8-48603c919b11 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-a8232ef0-e21d-45a9-83a8-48603c919b11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:12.632357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:12.633315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:12.634217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:12.634217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:12.634217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:12.634217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 296/585] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:12.635396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8232ef0-e21d-45a9-83a8-48603c919b11 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.642473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:12.653220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd92443-31a0-4658-8fe9-1501fdc8e4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.654381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd92443-31a0-4658-8fe9-1501fdc8e4bc None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.654854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd92443-31a0-4658-8fe9-1501fdc8e4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.655441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd92443-31a0-4658-8fe9-1501fdc8e4bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.669262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd92443-31a0-4658-8fe9-1501fdc8e4bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.672209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd92443-31a0-4658-8fe9-1501fdc8e4bc None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-ffd92443-31a0-4658-8fe9-1501fdc8e4bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:12.692263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:12.693230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:12.694101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:12.694101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:12.694101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:12.696498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 290/586] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:12.697412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd92443-31a0-4658-8fe9-1501fdc8e4bc None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.738148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:12.761363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8232ef0-e21d-45a9-83a8-48603c919b11 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:12.800180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e90e89-c286-4bfc-bbb5-a5118796ea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.801436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e90e89-c286-4bfc-bbb5-a5118796ea58 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.801814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e90e89-c286-4bfc-bbb5-a5118796ea58 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.802147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e90e89-c286-4bfc-bbb5-a5118796ea58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.809608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd92443-31a0-4658-8fe9-1501fdc8e4bc None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:12.814389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e90e89-c286-4bfc-bbb5-a5118796ea58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.815902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e90e89-c286-4bfc-bbb5-a5118796ea58 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-e4e90e89-c286-4bfc-bbb5-a5118796ea58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:12.831876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:12.833047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:12.834147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:12.834147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:12.834147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:12.837638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 297/587] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:12.838607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e90e89-c286-4bfc-bbb5-a5118796ea58 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.855560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7f72e-4ad3-45c9-b216-f97cf86a9bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:12.856190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7f72e-4ad3-45c9-b216-f97cf86a9bce None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:12.856583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7f72e-4ad3-45c9-b216-f97cf86a9bce None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:12.857043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7f72e-4ad3-45c9-b216-f97cf86a9bce None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:12.858172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7f72e-4ad3-45c9-b216-f97cf86a9bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:12.858487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7f72e-4ad3-45c9-b216-f97cf86a9bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:12.860179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7f72e-4ad3-45c9-b216-f97cf86a9bce None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:12.875787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 291/588] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:12.877949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7f72e-4ad3-45c9-b216-f97cf86a9bce None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:12.936396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:12.964400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e90e89-c286-4bfc-bbb5-a5118796ea58 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.004465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:13.013176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d8fbe-43f3-4c31-9bd0-6a0ec92aa729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.014175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d8fbe-43f3-4c31-9bd0-6a0ec92aa729 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.014376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7f72e-4ad3-45c9-b216-f97cf86a9bce None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.014889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d8fbe-43f3-4c31-9bd0-6a0ec92aa729 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.015363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d8fbe-43f3-4c31-9bd0-6a0ec92aa729 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.016311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d8fbe-43f3-4c31-9bd0-6a0ec92aa729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:13.016959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d8fbe-43f3-4c31-9bd0-6a0ec92aa729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.018572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d8fbe-43f3-4c31-9bd0-6a0ec92aa729 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.038154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 298/589] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:13.040129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d8fbe-43f3-4c31-9bd0-6a0ec92aa729 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.050372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf5f34-4b25-4f68-8a29-80ddfe5bd045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.050974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf5f34-4b25-4f68-8a29-80ddfe5bd045 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.051312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf5f34-4b25-4f68-8a29-80ddfe5bd045 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.051583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf5f34-4b25-4f68-8a29-80ddfe5bd045 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.052390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf5f34-4b25-4f68-8a29-80ddfe5bd045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:13.052500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf5f34-4b25-4f68-8a29-80ddfe5bd045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.053899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf5f34-4b25-4f68-8a29-80ddfe5bd045 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.070444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 292/590] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:13.070444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf5f34-4b25-4f68-8a29-80ddfe5bd045 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.171320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:13.175108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf5f34-4b25-4f68-8a29-80ddfe5bd045 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.178031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d8fbe-43f3-4c31-9bd0-6a0ec92aa729 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.213152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44ee5f-c525-4e93-bc7d-c9abf1bc6a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.213970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44ee5f-c525-4e93-bc7d-c9abf1bc6a7b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.214251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44ee5f-c525-4e93-bc7d-c9abf1bc6a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.214519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44ee5f-c525-4e93-bc7d-c9abf1bc6a7b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.222459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44ee5f-c525-4e93-bc7d-c9abf1bc6a7b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.222709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9d036-e79e-4fff-96d3-da878206d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.223569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9d036-e79e-4fff-96d3-da878206d481 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.223975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44ee5f-c525-4e93-bc7d-c9abf1bc6a7b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.224255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9d036-e79e-4fff-96d3-da878206d481 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.224721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9d036-e79e-4fff-96d3-da878206d481 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.233390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9d036-e79e-4fff-96d3-da878206d481 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.235083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9d036-e79e-4fff-96d3-da878206d481 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-ef44ee5f-c525-4e93-bc7d-c9abf1bc6a7b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:13.236741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:13.237663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 293/591] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:13.243535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44ee5f-c525-4e93-bc7d-c9abf1bc6a7b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-1ef9d036-e79e-4fff-96d3-da878206d481 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:13.269786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:13.270932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 20:09:13.272052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:13.274706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 299/592] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:13.275618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9d036-e79e-4fff-96d3-da878206d481 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.344303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:13.363327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44ee5f-c525-4e93-bc7d-c9abf1bc6a7b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.400652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673df22-6848-44bb-aa23-961b847064c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.402054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673df22-6848-44bb-aa23-961b847064c7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.402575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673df22-6848-44bb-aa23-961b847064c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.402982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673df22-6848-44bb-aa23-961b847064c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.405691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9d036-e79e-4fff-96d3-da878206d481 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.410687 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673df22-6848-44bb-aa23-961b847064c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.412283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673df22-6848-44bb-aa23-961b847064c7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-c673df22-6848-44bb-aa23-961b847064c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:13.428769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:13.429820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:13.430709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:13.430709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:13.430709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:13.430709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:13.430709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:13.430709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:13.430709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:13.430709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:13.432885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 294/593] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:13.433825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673df22-6848-44bb-aa23-961b847064c7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.451283 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736bf37-b009-4919-93b7-7b4885bee9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.452351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736bf37-b009-4919-93b7-7b4885bee9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.452652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736bf37-b009-4919-93b7-7b4885bee9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.452927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736bf37-b009-4919-93b7-7b4885bee9c7 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.453966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736bf37-b009-4919-93b7-7b4885bee9c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:13.454302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736bf37-b009-4919-93b7-7b4885bee9c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.456561 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736bf37-b009-4919-93b7-7b4885bee9c7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.461570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:13.481122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 300/594] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 20:09:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:13.486337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736bf37-b009-4919-93b7-7b4885bee9c7 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.557225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673df22-6848-44bb-aa23-961b847064c7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.605683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736bf37-b009-4919-93b7-7b4885bee9c7 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.605905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ddf52-8745-43b1-adcb-d8f29936bdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.606331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ddf52-8745-43b1-adcb-d8f29936bdbf None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.606608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ddf52-8745-43b1-adcb-d8f29936bdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.606879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ddf52-8745-43b1-adcb-d8f29936bdbf None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.608740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ddf52-8745-43b1-adcb-d8f29936bdbf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:13.609060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ddf52-8745-43b1-adcb-d8f29936bdbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.610620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ddf52-8745-43b1-adcb-d8f29936bdbf None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.617951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:13.639552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 295/595] 10.222.0.23 () {56 vars in 1139 bytes} [Wed Oct 5 20:09:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:13.640842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ddf52-8745-43b1-adcb-d8f29936bdbf None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.648409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c6355-b056-48ce-8813-b1793ba2605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.649066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c6355-b056-48ce-8813-b1793ba2605e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.649351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c6355-b056-48ce-8813-b1793ba2605e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.649662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c6355-b056-48ce-8813-b1793ba2605e None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.650408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c6355-b056-48ce-8813-b1793ba2605e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:13.650710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c6355-b056-48ce-8813-b1793ba2605e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.652180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c6355-b056-48ce-8813-b1793ba2605e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.664998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 301/596] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:13.665930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c6355-b056-48ce-8813-b1793ba2605e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.742725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:13.772504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ddf52-8745-43b1-adcb-d8f29936bdbf None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.772821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c6355-b056-48ce-8813-b1793ba2605e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.808964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c624-561b-4bc2-903d-5b713276fe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.809626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c624-561b-4bc2-903d-5b713276fe9d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.809909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c624-561b-4bc2-903d-5b713276fe9d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.810180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c624-561b-4bc2-903d-5b713276fe9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.820371 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e139b0-db3b-4f51-8b11-58443363a215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:13.821283 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c624-561b-4bc2-903d-5b713276fe9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.821411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e139b0-db3b-4f51-8b11-58443363a215 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:13.821850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e139b0-db3b-4f51-8b11-58443363a215 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:13.822275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e139b0-db3b-4f51-8b11-58443363a215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:13.823127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8c624-561b-4bc2-903d-5b713276fe9d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.835518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e139b0-db3b-4f51-8b11-58443363a215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:13.837218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e139b0-db3b-4f51-8b11-58443363a215 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-27f8c624-561b-4bc2-903d-5b713276fe9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:13.838195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:13.839316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:13.840408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:13.840408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:13.840408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:13.846683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 302/597] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:13.846683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c624-561b-4bc2-903d-5b713276fe9d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-24e139b0-db3b-4f51-8b11-58443363a215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:13.858670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:13.859615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:13.860565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:13.860565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:13.860565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:13.860565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 296/598] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:13.860800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e139b0-db3b-4f51-8b11-58443363a215 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:13.878750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:13.977251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c624-561b-4bc2-903d-5b713276fe9d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:13.989938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e139b0-db3b-4f51-8b11-58443363a215 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:14.020086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7d3a0-db3d-48da-8bd8-fc36f5bcbc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:14.020662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7d3a0-db3d-48da-8bd8-fc36f5bcbc54 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.020943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7d3a0-db3d-48da-8bd8-fc36f5bcbc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.021219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7d3a0-db3d-48da-8bd8-fc36f5bcbc54 None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/47bafea2367e4c7992354bb2c23e7d57/roles/89ae1151daf54ba9b292e812aac0ecc6` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.022289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:14.029808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0be244-d21f-401e-bd86-a004c37d2d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:14.030719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0be244-d21f-401e-bd86-a004c37d2d89 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.031214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0be244-d21f-401e-bd86-a004c37d2d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.031691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0be244-d21f-401e-bd86-a004c37d2d89 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.032567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0be244-d21f-401e-bd86-a004c37d2d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:14.033081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0be244-d21f-401e-bd86-a004c37d2d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:14.034685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0be244-d21f-401e-bd86-a004c37d2d89 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:14.039750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7d3a0-db3d-48da-8bd8-fc36f5bcbc54 None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=47bafea2367e4c7992354bb2c23e7d57, role_id=89ae1151daf54ba9b292e812aac0ecc6)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:14.041195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7d3a0-db3d-48da-8bd8-fc36f5bcbc54 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:14.049559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 297/599] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:14.051255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0be244-d21f-401e-bd86-a004c37d2d89 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:14.061274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 303/600] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:09:13 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/47bafea2367e4c7992354bb2c23e7d57/roles/89ae1151daf54ba9b292e812aac0ecc6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:09:14.062427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7d3a0-db3d-48da-8bd8-fc36f5bcbc54 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:14.264136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:14.323323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0be244-d21f-401e-bd86-a004c37d2d89 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:14.339905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7d3a0-db3d-48da-8bd8-fc36f5bcbc54 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:14.388237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b05a5d-8bde-497f-8fb2-eca2a8e07ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:14.388600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b05a5d-8bde-497f-8fb2-eca2a8e07ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.388834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b05a5d-8bde-497f-8fb2-eca2a8e07ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.389100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b05a5d-8bde-497f-8fb2-eca2a8e07ce2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.401656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11486f-9e53-49a5-a1b9-f696166eaad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:14.402350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11486f-9e53-49a5-a1b9-f696166eaad6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.402631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11486f-9e53-49a5-a1b9-f696166eaad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.402899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11486f-9e53-49a5-a1b9-f696166eaad6 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.403693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11486f-9e53-49a5-a1b9-f696166eaad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:14.403999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11486f-9e53-49a5-a1b9-f696166eaad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:14.405459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11486f-9e53-49a5-a1b9-f696166eaad6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:14.418334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b05a5d-8bde-497f-8fb2-eca2a8e07ce2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:14.418847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 298/601] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:14.419788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b05a5d-8bde-497f-8fb2-eca2a8e07ce2 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:14.424771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11486f-9e53-49a5-a1b9-f696166eaad6 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-f0b05a5d-8bde-497f-8fb2-eca2a8e07ce2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:14.449609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:14.450537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:14.451399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:14.451399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:14.451399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:14.451399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:14.451399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:14.451399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:14.451399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:14.451399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:14.456310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 304/602] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:14.468802 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b05a5d-8bde-497f-8fb2-eca2a8e07ce2 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:14.558607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:14.639584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b05a5d-8bde-497f-8fb2-eca2a8e07ce2 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:14.651266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11486f-9e53-49a5-a1b9-f696166eaad6 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:14.677185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:14.682542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefa5b8-2cb2-4bd2-8580-a80686a7e323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:14.683154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefa5b8-2cb2-4bd2-8580-a80686a7e323 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.683425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefa5b8-2cb2-4bd2-8580-a80686a7e323 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.683690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefa5b8-2cb2-4bd2-8580-a80686a7e323 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.684374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefa5b8-2cb2-4bd2-8580-a80686a7e323 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:14.684671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefa5b8-2cb2-4bd2-8580-a80686a7e323 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:14.686130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefa5b8-2cb2-4bd2-8580-a80686a7e323 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:14.692347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fa500-9e9b-49e2-a772-30071f669c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:14.692975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fa500-9e9b-49e2-a772-30071f669c1a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.693255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fa500-9e9b-49e2-a772-30071f669c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.694034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fa500-9e9b-49e2-a772-30071f669c1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.698591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 305/603] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:14.699615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a2e05-e99c-4b15-8a4f-7e0d77cd7c8f None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.699894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a2e05-e99c-4b15-8a4f-7e0d77cd7c8f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.700158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a2e05-e99c-4b15-8a4f-7e0d77cd7c8f None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.701456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 306/604] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:14.701588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653fa500-9e9b-49e2-a772-30071f669c1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:14.703177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653fa500-9e9b-49e2-a772-30071f669c1a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:14.707023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9195f-362e-40ed-9822-d42301378736 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.707307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9195f-362e-40ed-9822-d42301378736 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.707581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9195f-362e-40ed-9822-d42301378736 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.708928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 307/605] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:14.709479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9195f-362e-40ed-9822-d42301378736 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-653fa500-9e9b-49e2-a772-30071f669c1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:14.719438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:14.721038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:14.721995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:14.721995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:14.721995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:14.721995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:14.721995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:14.721995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:14.721995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:14.721995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:14.725722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 299/606] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:14.727378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568652-0647-4813-b3a0-72d847d37471 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.727712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568652-0647-4813-b3a0-72d847d37471 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.727993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568652-0647-4813-b3a0-72d847d37471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.747108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c3568652-0647-4813-b3a0-72d847d37471 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:14.748312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c3568652-0647-4813-b3a0-72d847d37471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:14.819351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 300/607] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:14.821536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8859b7c-f990-44c7-b8e7-285b7255e52c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.822186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8859b7c-f990-44c7-b8e7-285b7255e52c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.822997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8859b7c-f990-44c7-b8e7-285b7255e52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.830877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9195f-362e-40ed-9822-d42301378736 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:14.848723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c8859b7c-f990-44c7-b8e7-285b7255e52c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:14.850756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c8859b7c-f990-44c7-b8e7-285b7255e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:14.874201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed31be1-086e-46c4-95db-9f7fa5d95737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:14.874887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed31be1-086e-46c4-95db-9f7fa5d95737 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:14.875163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed31be1-086e-46c4-95db-9f7fa5d95737 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:14.875434 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed31be1-086e-46c4-95db-9f7fa5d95737 None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/8f5a88f263014f15a6f59cfdaa2bb6d8/roles/89ae1151daf54ba9b292e812aac0ecc6` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:14.895987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed31be1-086e-46c4-95db-9f7fa5d95737 None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=8f5a88f263014f15a6f59cfdaa2bb6d8, role_id=89ae1151daf54ba9b292e812aac0ecc6)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:14.898524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed31be1-086e-46c4-95db-9f7fa5d95737 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:14.904625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:14.918182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 308/608] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:09:14 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/8f5a88f263014f15a6f59cfdaa2bb6d8/roles/89ae1151daf54ba9b292e812aac0ecc6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:09:14.919522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed31be1-086e-46c4-95db-9f7fa5d95737 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:14.945185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 301/609] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:14.974473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:15.038917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed31be1-086e-46c4-95db-9f7fa5d95737 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:15.076549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4d0f1-da2d-4e00-95d5-a5c6c0c461cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:15.077612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4d0f1-da2d-4e00-95d5-a5c6c0c461cd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:15.078092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4d0f1-da2d-4e00-95d5-a5c6c0c461cd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:15.078652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4d0f1-da2d-4e00-95d5-a5c6c0c461cd None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:15.080103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4d0f1-da2d-4e00-95d5-a5c6c0c461cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:15.080636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4d0f1-da2d-4e00-95d5-a5c6c0c461cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:15.082321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4d0f1-da2d-4e00-95d5-a5c6c0c461cd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:15.096728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 309/610] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:15.102748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8859b7c-f990-44c7-b8e7-285b7255e52c None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:15.128036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4d0f1-da2d-4e00-95d5-a5c6c0c461cd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:15.249305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4d0f1-da2d-4e00-95d5-a5c6c0c461cd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:15.251561 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:15.253756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8859b7c-f990-44c7-b8e7-285b7255e52c None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:15.298464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846dad-e2ce-4758-8eba-cd4f9015a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:15.299096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846dad-e2ce-4758-8eba-cd4f9015a210 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:15.299369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846dad-e2ce-4758-8eba-cd4f9015a210 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:15.299637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846dad-e2ce-4758-8eba-cd4f9015a210 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:15.305215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f71da-e662-43f4-aed3-e24391011771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:15.305973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f71da-e662-43f4-aed3-e24391011771 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:15.306251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f71da-e662-43f4-aed3-e24391011771 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:15.306522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f71da-e662-43f4-aed3-e24391011771 None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/a6e8bc66986d48aa9dcbc19258ff6e1f/roles/89ae1151daf54ba9b292e812aac0ecc6` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:15.311886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf846dad-e2ce-4758-8eba-cd4f9015a210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-cf846dad-e2ce-4758-8eba-cd4f9015a210 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:15.319405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:15.320394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:15.321203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 310/611] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:15.321935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846dad-e2ce-4758-8eba-cd4f9015a210 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:15.336144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f71da-e662-43f4-aed3-e24391011771 None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=a6e8bc66986d48aa9dcbc19258ff6e1f, role_id=89ae1151daf54ba9b292e812aac0ecc6)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:15.338092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f71da-e662-43f4-aed3-e24391011771 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:15.356632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 302/612] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:09:15 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/a6e8bc66986d48aa9dcbc19258ff6e1f/roles/89ae1151daf54ba9b292e812aac0ecc6 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:09:15.357453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f71da-e662-43f4-aed3-e24391011771 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:15.403244 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:15.519810 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846dad-e2ce-4758-8eba-cd4f9015a210 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:15.561027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f71da-e662-43f4-aed3-e24391011771 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:15.591904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e17253-381e-4d96-a4f3-592dbc084dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:15.592536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e17253-381e-4d96-a4f3-592dbc084dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:15.592809 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e17253-381e-4d96-a4f3-592dbc084dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:15.593079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e17253-381e-4d96-a4f3-592dbc084dd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:15.606413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e17253-381e-4d96-a4f3-592dbc084dd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:15.607548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab64bb-56fd-45f7-9e3b-f32106e10b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:15.608170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab64bb-56fd-45f7-9e3b-f32106e10b85 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:15.608451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab64bb-56fd-45f7-9e3b-f32106e10b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:15.608769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab64bb-56fd-45f7-9e3b-f32106e10b85 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:15.609453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab64bb-56fd-45f7-9e3b-f32106e10b85 None admin] RBAC: Adding query filter params (name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:15.609785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab64bb-56fd-45f7-9e3b-f32106e10b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:15.611201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab64bb-56fd-45f7-9e3b-f32106e10b85 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-01e17253-381e-4d96-a4f3-592dbc084dd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:15.612500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:15.613568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:15.614377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 311/613] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:15.619288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e17253-381e-4d96-a4f3-592dbc084dd4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:15.623571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 303/614] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:15.629506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab64bb-56fd-45f7-9e3b-f32106e10b85 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:15.648301 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:15.726033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:15.743163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab64bb-56fd-45f7-9e3b-f32106e10b85 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:15.792354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38e115-834b-4fca-9747-0d5cc809ef84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:15.792915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38e115-834b-4fca-9747-0d5cc809ef84 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:15.793192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38e115-834b-4fca-9747-0d5cc809ef84 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:15.793464 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38e115-834b-4fca-9747-0d5cc809ef84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:15.801246 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e17253-381e-4d96-a4f3-592dbc084dd4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:15.818706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38e115-834b-4fca-9747-0d5cc809ef84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:15.820171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38e115-834b-4fca-9747-0d5cc809ef84 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:15.844337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3403660-3f3c-4db0-86f3-d57365ba9585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:15.844989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3403660-3f3c-4db0-86f3-d57365ba9585 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:15.845275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3403660-3f3c-4db0-86f3-d57365ba9585 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:15.845597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3403660-3f3c-4db0-86f3-d57365ba9585 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:15.846293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3403660-3f3c-4db0-86f3-d57365ba9585 None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:15.846594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3403660-3f3c-4db0-86f3-d57365ba9585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:15.847990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3403660-3f3c-4db0-86f3-d57365ba9585 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-2d38e115-834b-4fca-9747-0d5cc809ef84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:15.851852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:15.852754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:15.853638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:15.853638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:15.853638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:15.853638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 304/615] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:15.856797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 312/616] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:15.860159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38e115-834b-4fca-9747-0d5cc809ef84 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:15.865634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3403660-3f3c-4db0-86f3-d57365ba9585 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:15.990190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:16.023205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3403660-3f3c-4db0-86f3-d57365ba9585 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:16.076304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38e115-834b-4fca-9747-0d5cc809ef84 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:16.090776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722ea56-48a1-4710-8b1b-6ade240d4085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:16.091420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722ea56-48a1-4710-8b1b-6ade240d4085 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:16.091701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722ea56-48a1-4710-8b1b-6ade240d4085 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:16.091974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722ea56-48a1-4710-8b1b-6ade240d4085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:16.114936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722ea56-48a1-4710-8b1b-6ade240d4085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:16.116396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722ea56-48a1-4710-8b1b-6ade240d4085 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:16.142223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4e032-281d-46c6-aa41-8f6e01f38d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:16.142893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4e032-281d-46c6-aa41-8f6e01f38d44 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:16.143171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4e032-281d-46c6-aa41-8f6e01f38d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:16.143441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4e032-281d-46c6-aa41-8f6e01f38d44 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-b722ea56-48a1-4710-8b1b-6ade240d4085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:16.145492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:16.146402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:16.147250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:16.147250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:16.147250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:16.147413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4e032-281d-46c6-aa41-8f6e01f38d44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:16.147413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4e032-281d-46c6-aa41-8f6e01f38d44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:16.148547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4e032-281d-46c6-aa41-8f6e01f38d44 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:16.150243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:16.154237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 313/617] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:16.156581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722ea56-48a1-4710-8b1b-6ade240d4085 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:16.170480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 305/618] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:16.176598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4e032-281d-46c6-aa41-8f6e01f38d44 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:16.270154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722ea56-48a1-4710-8b1b-6ade240d4085 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:16.305409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461db392-b29e-470c-b009-d05ce6506230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:16.306047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db392-b29e-470c-b009-d05ce6506230 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:16.306328 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db392-b29e-470c-b009-d05ce6506230 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:16.306601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461db392-b29e-470c-b009-d05ce6506230 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:16.307243 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461db392-b29e-470c-b009-d05ce6506230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:16.307549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461db392-b29e-470c-b009-d05ce6506230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:16.308934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461db392-b29e-470c-b009-d05ce6506230 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:16.311399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:16.322854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 314/619] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:16.328314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461db392-b29e-470c-b009-d05ce6506230 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:16.434552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461db392-b29e-470c-b009-d05ce6506230 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:16.450492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4e032-281d-46c6-aa41-8f6e01f38d44 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:16.472466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58319c21-126b-49f6-be79-0e67f1406c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:16.473104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58319c21-126b-49f6-be79-0e67f1406c69 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:16.473388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58319c21-126b-49f6-be79-0e67f1406c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:16.473674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58319c21-126b-49f6-be79-0e67f1406c69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:16.480970 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58319c21-126b-49f6-be79-0e67f1406c69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:16.482578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58319c21-126b-49f6-be79-0e67f1406c69 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-58319c21-126b-49f6-be79-0e67f1406c69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:16.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:16.495440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:16.496361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 315/620] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 20:09:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:16.501833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58319c21-126b-49f6-be79-0e67f1406c69 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:16.550427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a113c-d5a2-4087-b4ad-500e9e520d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:16.551063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a113c-d5a2-4087-b4ad-500e9e520d4c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:16.551391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a113c-d5a2-4087-b4ad-500e9e520d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:16.551671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a113c-d5a2-4087-b4ad-500e9e520d4c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:16.560020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a113c-d5a2-4087-b4ad-500e9e520d4c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:16.561611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a113c-d5a2-4087-b4ad-500e9e520d4c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-040a113c-d5a2-4087-b4ad-500e9e520d4c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:16.576254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:16.577210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:16.578140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 306/621] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:16.583836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a113c-d5a2-4087-b4ad-500e9e520d4c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:16.597222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:16.630987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58319c21-126b-49f6-be79-0e67f1406c69 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:16.670100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23135148-246d-4997-94e0-043ba0d7c450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:16.670917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23135148-246d-4997-94e0-043ba0d7c450 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:16.671103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23135148-246d-4997-94e0-043ba0d7c450 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:16.671358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23135148-246d-4997-94e0-043ba0d7c450 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:16.672106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23135148-246d-4997-94e0-043ba0d7c450 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:16.672417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23135148-246d-4997-94e0-043ba0d7c450 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:16.674522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23135148-246d-4997-94e0-043ba0d7c450 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:16.695367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 316/622] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 20:09:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:16.700878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23135148-246d-4997-94e0-043ba0d7c450 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:16.754792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:16.804418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23135148-246d-4997-94e0-043ba0d7c450 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:16.813579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a113c-d5a2-4087-b4ad-500e9e520d4c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:16.819410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:16.840347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6875d1-fa52-4d70-8c69-6327084899a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:16.840958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6875d1-fa52-4d70-8c69-6327084899a6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:16.841240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6875d1-fa52-4d70-8c69-6327084899a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:16.841531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6875d1-fa52-4d70-8c69-6327084899a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:16.853469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14ba35-8c78-4523-94a2-bb1a166a89b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:16.854517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14ba35-8c78-4523-94a2-bb1a166a89b0 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:16.854996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14ba35-8c78-4523-94a2-bb1a166a89b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:16.855456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14ba35-8c78-4523-94a2-bb1a166a89b0 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:16.856433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14ba35-8c78-4523-94a2-bb1a166a89b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:16.857056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14ba35-8c78-4523-94a2-bb1a166a89b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:16.858830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14ba35-8c78-4523-94a2-bb1a166a89b0 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:16.880064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 307/623] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 20:09:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:16.881781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb5efb-c979-4f22-a91e-0f0b79c96e35 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:16.882382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb5efb-c979-4f22-a91e-0f0b79c96e35 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:16.882981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb5efb-c979-4f22-a91e-0f0b79c96e35 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:16.888912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 308/624] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:16.890099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb5efb-c979-4f22-a91e-0f0b79c96e35 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:16.921571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6875d1-fa52-4d70-8c69-6327084899a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:16.923072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6875d1-fa52-4d70-8c69-6327084899a6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:17.007520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb5efb-c979-4f22-a91e-0f0b79c96e35 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:17.020771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:17.031307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 317/625] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 20:09:17.033539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f61c8f-a67f-43cc-b7c1-c81a6a6f01b2 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.034023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f61c8f-a67f-43cc-b7c1-c81a6a6f01b2 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.034487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f61c8f-a67f-43cc-b7c1-c81a6a6f01b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.050329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fe054-ae8a-4129-8312-9a81827a8c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:17.051271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fe054-ae8a-4129-8312-9a81827a8c86 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.051799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fe054-ae8a-4129-8312-9a81827a8c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.052264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fe054-ae8a-4129-8312-9a81827a8c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.055402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-18f61c8f-a67f-43cc-b7c1-c81a6a6f01b2 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:17.056599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-18f61c8f-a67f-43cc-b7c1-c81a6a6f01b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:17.063815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fe054-ae8a-4129-8312-9a81827a8c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:17.065929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fe054-ae8a-4129-8312-9a81827a8c86 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:17.066410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-ce2fe054-ae8a-4129-8312-9a81827a8c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:17.084325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:17.085311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:17.086200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:17.086200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:17.086200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:17.088308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 309/626] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:17.089419 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fe054-ae8a-4129-8312-9a81827a8c86 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:17.129651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 318/627] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:17.129651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f61c8f-a67f-43cc-b7c1-c81a6a6f01b2 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:17.223625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fe054-ae8a-4129-8312-9a81827a8c86 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:17.275223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e0cd7-a7b4-44bc-8d1c-5ed7ba6d05fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:17.275855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e0cd7-a7b4-44bc-8d1c-5ed7ba6d05fd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.276128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e0cd7-a7b4-44bc-8d1c-5ed7ba6d05fd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.276395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e0cd7-a7b4-44bc-8d1c-5ed7ba6d05fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.297668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e0cd7-a7b4-44bc-8d1c-5ed7ba6d05fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:17.299646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e0cd7-a7b4-44bc-8d1c-5ed7ba6d05fd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:17.316208 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f61c8f-a67f-43cc-b7c1-c81a6a6f01b2 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-445e0cd7-a7b4-44bc-8d1c-5ed7ba6d05fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:17.328741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:17.329740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:17.331260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:17.331260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:17.331260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:17.331260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 310/628] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:17.336396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e0cd7-a7b4-44bc-8d1c-5ed7ba6d05fd None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:17.367010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19f979-df11-4209-a07b-854db06a2b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:17.367733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19f979-df11-4209-a07b-854db06a2b2e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.368013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19f979-df11-4209-a07b-854db06a2b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.368283 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19f979-df11-4209-a07b-854db06a2b2e None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.368921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19f979-df11-4209-a07b-854db06a2b2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:17.369217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19f979-df11-4209-a07b-854db06a2b2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:17.370719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19f979-df11-4209-a07b-854db06a2b2e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:17.377930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:17.385388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 319/629] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:17.390483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19f979-df11-4209-a07b-854db06a2b2e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:17.395551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:17.522760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e0cd7-a7b4-44bc-8d1c-5ed7ba6d05fd None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:17.540305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19f979-df11-4209-a07b-854db06a2b2e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:17.563826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536c935-21c4-499e-9b83-17048a1c627c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:17.564704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536c935-21c4-499e-9b83-17048a1c627c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.564979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536c935-21c4-499e-9b83-17048a1c627c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.565251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536c935-21c4-499e-9b83-17048a1c627c None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.565981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536c935-21c4-499e-9b83-17048a1c627c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:17.566285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536c935-21c4-499e-9b83-17048a1c627c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:17.567699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536c935-21c4-499e-9b83-17048a1c627c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:17.575956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163c4e8-2e4a-4b33-ab0d-0de09d2dc31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:17.576579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163c4e8-2e4a-4b33-ab0d-0de09d2dc31c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.576852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163c4e8-2e4a-4b33-ab0d-0de09d2dc31c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.577122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163c4e8-2e4a-4b33-ab0d-0de09d2dc31c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.581398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 311/630] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:17.582484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536c935-21c4-499e-9b83-17048a1c627c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:17.586861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163c4e8-2e4a-4b33-ab0d-0de09d2dc31c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:17.588295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163c4e8-2e4a-4b33-ab0d-0de09d2dc31c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-6163c4e8-2e4a-4b33-ab0d-0de09d2dc31c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:17.600354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:17.601675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 320/631] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:17.602548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9891854-9f26-47a9-a0e1-ea70e81e29c1 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.603072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9891854-9f26-47a9-a0e1-ea70e81e29c1 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.603072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9891854-9f26-47a9-a0e1-ea70e81e29c1 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.608084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 321/632] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:17.608084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9891854-9f26-47a9-a0e1-ea70e81e29c1 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:17.672016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:17.673351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:17.699078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536c935-21c4-499e-9b83-17048a1c627c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:17.714275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9891854-9f26-47a9-a0e1-ea70e81e29c1 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:17.739446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab341a5-eabe-4e8f-8286-8fe923cd7f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:17.740389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab341a5-eabe-4e8f-8286-8fe923cd7f2c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.740671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab341a5-eabe-4e8f-8286-8fe923cd7f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.740940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab341a5-eabe-4e8f-8286-8fe923cd7f2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.752675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27cf65-9f8f-4a13-8980-4603d0297278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:17.752911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab341a5-eabe-4e8f-8286-8fe923cd7f2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:17.753697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27cf65-9f8f-4a13-8980-4603d0297278 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.754116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27cf65-9f8f-4a13-8980-4603d0297278 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.754527 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27cf65-9f8f-4a13-8980-4603d0297278 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-8ab341a5-eabe-4e8f-8286-8fe923cd7f2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:17.757707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:17.758792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:17.759627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 312/633] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:17.760297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab341a5-eabe-4e8f-8286-8fe923cd7f2c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:17.764940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27cf65-9f8f-4a13-8980-4603d0297278 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:17.766585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e27cf65-9f8f-4a13-8980-4603d0297278 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-0e27cf65-9f8f-4a13-8980-4603d0297278 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:17.779838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:17.780905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:17.781955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:17.781955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:17.781955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:17.781955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:17.781955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:17.781955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:17.781955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:17.781955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:17.784150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 322/634] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:17.785408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051843-2f62-4d7f-84b0-78759c1da2f0 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.785854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051843-2f62-4d7f-84b0-78759c1da2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.786123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051843-2f62-4d7f-84b0-78759c1da2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.804883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c4051843-2f62-4d7f-84b0-78759c1da2f0 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:17.806096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c4051843-2f62-4d7f-84b0-78759c1da2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:17.870163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab341a5-eabe-4e8f-8286-8fe923cd7f2c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:17.875856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 323/635] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:17.876555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051843-2f62-4d7f-84b0-78759c1da2f0 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:17.911269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752af3-38c6-4b5c-b817-82d0213ca203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:17.911866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752af3-38c6-4b5c-b817-82d0213ca203 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:17.912143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752af3-38c6-4b5c-b817-82d0213ca203 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:17.912408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb752af3-38c6-4b5c-b817-82d0213ca203 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:17.913231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb752af3-38c6-4b5c-b817-82d0213ca203 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:17.913562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb752af3-38c6-4b5c-b817-82d0213ca203 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:17.914963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb752af3-38c6-4b5c-b817-82d0213ca203 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:17.918045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:17.928221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 313/636] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:17.928954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752af3-38c6-4b5c-b817-82d0213ca203 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:17.935862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:17.985168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051843-2f62-4d7f-84b0-78759c1da2f0 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.020355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76ca62-f546-4e4e-aeb0-6e47caef84ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.020967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76ca62-f546-4e4e-aeb0-6e47caef84ff None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.021241 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76ca62-f546-4e4e-aeb0-6e47caef84ff None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.021528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76ca62-f546-4e4e-aeb0-6e47caef84ff None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.022266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76ca62-f546-4e4e-aeb0-6e47caef84ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:18.022544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76ca62-f546-4e4e-aeb0-6e47caef84ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:18.023898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76ca62-f546-4e4e-aeb0-6e47caef84ff None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:18.031626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 324/637] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:18.033361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7877a-ac20-46ec-8091-903597ebdff8 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.033671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7877a-ac20-46ec-8091-903597ebdff8 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.033939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7877a-ac20-46ec-8091-903597ebdff8 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.034930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb752af3-38c6-4b5c-b817-82d0213ca203 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.037628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 325/638] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:18.037628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7877a-ac20-46ec-8091-903597ebdff8 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:18.073657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e01f6-ebce-402e-b0ad-af7070952986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.074864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e01f6-ebce-402e-b0ad-af7070952986 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.075291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e01f6-ebce-402e-b0ad-af7070952986 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.076058 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e01f6-ebce-402e-b0ad-af7070952986 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.076965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e01f6-ebce-402e-b0ad-af7070952986 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:18.077410 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e01f6-ebce-402e-b0ad-af7070952986 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:18.079677 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e01f6-ebce-402e-b0ad-af7070952986 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:18.096973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 314/639] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:18.097941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e01f6-ebce-402e-b0ad-af7070952986 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:18.152716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7877a-ac20-46ec-8091-903597ebdff8 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.153983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:18.169169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:18.186816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763bea7-585e-4e68-a27f-ff89acf7c47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.187392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763bea7-585e-4e68-a27f-ff89acf7c47c None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.187665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763bea7-585e-4e68-a27f-ff89acf7c47c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.187929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763bea7-585e-4e68-a27f-ff89acf7c47c None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/f8dec1dbdcd94316aad25c93c313a5a5/roles/89ae1151daf54ba9b292e812aac0ecc6` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.198221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e01f6-ebce-402e-b0ad-af7070952986 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.202136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763bea7-585e-4e68-a27f-ff89acf7c47c None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=f8dec1dbdcd94316aad25c93c313a5a5, role_id=89ae1151daf54ba9b292e812aac0ecc6)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:18.203542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763bea7-585e-4e68-a27f-ff89acf7c47c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:18.216235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 326/640] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:09:18 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/f8dec1dbdcd94316aad25c93c313a5a5/roles/89ae1151daf54ba9b292e812aac0ecc6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:09:18.217465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9df73-6492-47c3-a382-8bb7c7da9921 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.217771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9df73-6492-47c3-a382-8bb7c7da9921 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.218042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9df73-6492-47c3-a382-8bb7c7da9921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.234710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b38779-b994-4c7e-82d8-9e8e526bcce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.235375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b38779-b994-4c7e-82d8-9e8e526bcce8 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.235643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b38779-b994-4c7e-82d8-9e8e526bcce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.235911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b38779-b994-4c7e-82d8-9e8e526bcce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.237762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-08d9df73-6492-47c3-a382-8bb7c7da9921 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:18.238994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-08d9df73-6492-47c3-a382-8bb7c7da9921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:18.247308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b38779-b994-4c7e-82d8-9e8e526bcce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:18.249483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b38779-b994-4c7e-82d8-9e8e526bcce8 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-39b38779-b994-4c7e-82d8-9e8e526bcce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:18.266292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:18.267896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:18.268746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:18.268746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:18.268746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:18.268746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 315/641] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:18.268746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b38779-b994-4c7e-82d8-9e8e526bcce8 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:18.323182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 327/642] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:18.326630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9df73-6492-47c3-a382-8bb7c7da9921 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:18.400872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b38779-b994-4c7e-82d8-9e8e526bcce8 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.411813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:18.446882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e233f7-bf2a-4a81-a6d2-c1b7d7f1ed79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.448014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e233f7-bf2a-4a81-a6d2-c1b7d7f1ed79 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.448480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e233f7-bf2a-4a81-a6d2-c1b7d7f1ed79 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.449064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e233f7-bf2a-4a81-a6d2-c1b7d7f1ed79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.450312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e233f7-bf2a-4a81-a6d2-c1b7d7f1ed79 None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=286f4e203e7a4a55bca0a58d061af2bd) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:18.450818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e233f7-bf2a-4a81-a6d2-c1b7d7f1ed79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:18.452942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e233f7-bf2a-4a81-a6d2-c1b7d7f1ed79 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:18.461857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 316/643] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:18 2022] GET /identity/v3/role_assignments?user.id=286f4e203e7a4a55bca0a58d061af2bd&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:18.463055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e233f7-bf2a-4a81-a6d2-c1b7d7f1ed79 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:18.463372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:18.473833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9df73-6492-47c3-a382-8bb7c7da9921 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.519828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917611a4-79aa-4d3b-a8dd-e8d4012c9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.521780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917611a4-79aa-4d3b-a8dd-e8d4012c9f78 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.521780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917611a4-79aa-4d3b-a8dd-e8d4012c9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.521780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917611a4-79aa-4d3b-a8dd-e8d4012c9f78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.537147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917611a4-79aa-4d3b-a8dd-e8d4012c9f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-917611a4-79aa-4d3b-a8dd-e8d4012c9f78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:18.542682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:18.543715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:18.544598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 328/644] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:18.556481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917611a4-79aa-4d3b-a8dd-e8d4012c9f78 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:18.643445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e233f7-bf2a-4a81-a6d2-c1b7d7f1ed79 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.693552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529a66-c347-4c07-8b1f-907e404946e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.694176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529a66-c347-4c07-8b1f-907e404946e0 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.694452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529a66-c347-4c07-8b1f-907e404946e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.694931 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37529a66-c347-4c07-8b1f-907e404946e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.695895 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37529a66-c347-4c07-8b1f-907e404946e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:18.695895 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37529a66-c347-4c07-8b1f-907e404946e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:18.701833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37529a66-c347-4c07-8b1f-907e404946e0 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:18.716571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 317/645] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:18.717776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529a66-c347-4c07-8b1f-907e404946e0 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:18.729882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917611a4-79aa-4d3b-a8dd-e8d4012c9f78 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.754852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:18.776384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b6642-ded9-4b1c-942a-06185b680d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.777273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b6642-ded9-4b1c-942a-06185b680d37 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.778350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b6642-ded9-4b1c-942a-06185b680d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.778636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b6642-ded9-4b1c-942a-06185b680d37 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.779721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b6642-ded9-4b1c-942a-06185b680d37 None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:18.780031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b6642-ded9-4b1c-942a-06185b680d37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:18.781671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b6642-ded9-4b1c-942a-06185b680d37 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:18.792644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 329/646] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:18.793563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b6642-ded9-4b1c-942a-06185b680d37 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:18.819072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:18.834967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37529a66-c347-4c07-8b1f-907e404946e0 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.879729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cac772-cffe-4222-a46b-4f7bab4e46ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.880730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cac772-cffe-4222-a46b-4f7bab4e46ee None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.881008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cac772-cffe-4222-a46b-4f7bab4e46ee None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.881277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cac772-cffe-4222-a46b-4f7bab4e46ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.897949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cac772-cffe-4222-a46b-4f7bab4e46ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-80cac772-cffe-4222-a46b-4f7bab4e46ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:18.903118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:18.904011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:18.904959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 318/647] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:18.905708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cac772-cffe-4222-a46b-4f7bab4e46ee None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:18.922756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b6642-ded9-4b1c-942a-06185b680d37 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:18.961466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fd264-fea7-43ca-a246-9015fb28000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:18.962039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fd264-fea7-43ca-a246-9015fb28000f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:18.962455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fd264-fea7-43ca-a246-9015fb28000f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:18.962726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fd264-fea7-43ca-a246-9015fb28000f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:18.971165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fd264-fea7-43ca-a246-9015fb28000f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:18.972432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fd264-fea7-43ca-a246-9015fb28000f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-fd6fd264-fea7-43ca-a246-9015fb28000f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:18.984808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:18.987536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 330/648] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:18.988990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fd264-fea7-43ca-a246-9015fb28000f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.028900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:19.038941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cac772-cffe-4222-a46b-4f7bab4e46ee None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:19.074983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eecb4a-90bb-40d9-b0d4-f60b30c8b3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:19.075792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eecb4a-90bb-40d9-b0d4-f60b30c8b3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:19.076295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eecb4a-90bb-40d9-b0d4-f60b30c8b3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:19.076512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eecb4a-90bb-40d9-b0d4-f60b30c8b3a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:19.087186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eecb4a-90bb-40d9-b0d4-f60b30c8b3a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:19.088740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eecb4a-90bb-40d9-b0d4-f60b30c8b3a2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:19.101754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-f5eecb4a-90bb-40d9-b0d4-f60b30c8b3a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:19.104554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:19.105828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:19.107103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:19.107103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:19.107103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:19.108233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 319/649] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:19.109230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eecb4a-90bb-40d9-b0d4-f60b30c8b3a2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.134215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fd264-fea7-43ca-a246-9015fb28000f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:19.183329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c3731-0bb8-4ce1-b585-67d5e78bd0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:19.184209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c3731-0bb8-4ce1-b585-67d5e78bd0be None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:19.184879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c3731-0bb8-4ce1-b585-67d5e78bd0be None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:19.185296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c3731-0bb8-4ce1-b585-67d5e78bd0be None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:19.187106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c3731-0bb8-4ce1-b585-67d5e78bd0be None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:19.187558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c3731-0bb8-4ce1-b585-67d5e78bd0be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:19.189900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c3731-0bb8-4ce1-b585-67d5e78bd0be None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:19.201616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 331/650] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:19.202598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c3731-0bb8-4ce1-b585-67d5e78bd0be None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eecb4a-90bb-40d9-b0d4-f60b30c8b3a2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:19.259199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa8b3f-2e68-4f1e-92ad-2072d5f41578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:19.260062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa8b3f-2e68-4f1e-92ad-2072d5f41578 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:19.260499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa8b3f-2e68-4f1e-92ad-2072d5f41578 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:19.261038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa8b3f-2e68-4f1e-92ad-2072d5f41578 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:19.261788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa8b3f-2e68-4f1e-92ad-2072d5f41578 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:19.262180 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa8b3f-2e68-4f1e-92ad-2072d5f41578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:19.263804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa8b3f-2e68-4f1e-92ad-2072d5f41578 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:19.287780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 320/651] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 20:09:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:19.287780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa8b3f-2e68-4f1e-92ad-2072d5f41578 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.288564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:19.337580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c3731-0bb8-4ce1-b585-67d5e78bd0be None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:19.380540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404ee44-f50c-49c2-bf2a-c0766acc42d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:19.381417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404ee44-f50c-49c2-bf2a-c0766acc42d5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:19.381861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404ee44-f50c-49c2-bf2a-c0766acc42d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:19.382299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404ee44-f50c-49c2-bf2a-c0766acc42d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:19.383519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404ee44-f50c-49c2-bf2a-c0766acc42d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:19.383970 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404ee44-f50c-49c2-bf2a-c0766acc42d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:19.386035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d404ee44-f50c-49c2-bf2a-c0766acc42d5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:19.396458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa8b3f-2e68-4f1e-92ad-2072d5f41578 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:19.398743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:19.403375 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 332/652] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:19.404411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404ee44-f50c-49c2-bf2a-c0766acc42d5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.444290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24677afd-dbd3-4636-935d-8ffcdb3e5620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:19.445332 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24677afd-dbd3-4636-935d-8ffcdb3e5620 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:19.445736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24677afd-dbd3-4636-935d-8ffcdb3e5620 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:19.446103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24677afd-dbd3-4636-935d-8ffcdb3e5620 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:19.460601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24677afd-dbd3-4636-935d-8ffcdb3e5620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:19.462982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24677afd-dbd3-4636-935d-8ffcdb3e5620 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-24677afd-dbd3-4636-935d-8ffcdb3e5620 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:19.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:19.482840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:19.483746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:19.483746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:19.483746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:19.489675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 321/653] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:19.490692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24677afd-dbd3-4636-935d-8ffcdb3e5620 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.522793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404ee44-f50c-49c2-bf2a-c0766acc42d5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:19.548746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:19.559358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000fdfc-f77b-45b2-8743-ed217384f0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:19.559994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000fdfc-f77b-45b2-8743-ed217384f0ec None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:19.560279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000fdfc-f77b-45b2-8743-ed217384f0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:19.560554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000fdfc-f77b-45b2-8743-ed217384f0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:19.601890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24677afd-dbd3-4636-935d-8ffcdb3e5620 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:19.632063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:19.634891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000fdfc-f77b-45b2-8743-ed217384f0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:19.636498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000fdfc-f77b-45b2-8743-ed217384f0ec None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:19.636920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccb9ec-ce00-44ee-8411-a609a5ef137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:19.637578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccb9ec-ce00-44ee-8411-a609a5ef137d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:19.637861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccb9ec-ce00-44ee-8411-a609a5ef137d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:19.638134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccb9ec-ce00-44ee-8411-a609a5ef137d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:19.646654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccb9ec-ce00-44ee-8411-a609a5ef137d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:19.648094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccb9ec-ce00-44ee-8411-a609a5ef137d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-c3ccb9ec-ce00-44ee-8411-a609a5ef137d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:19.660089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:19.661209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 322/654] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:19.666538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccb9ec-ce00-44ee-8411-a609a5ef137d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.750092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 333/655] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:09:19.751548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000fdfc-f77b-45b2-8743-ed217384f0ec None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.771241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccb9ec-ce00-44ee-8411-a609a5ef137d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:19.785876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:19.807852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f8e3f-90f5-4606-a10a-b784295d8819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:19.808529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f8e3f-90f5-4606-a10a-b784295d8819 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:19.808731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f8e3f-90f5-4606-a10a-b784295d8819 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:19.809034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f8e3f-90f5-4606-a10a-b784295d8819 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:19.809665 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f8e3f-90f5-4606-a10a-b784295d8819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:19.809964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f8e3f-90f5-4606-a10a-b784295d8819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:19.811328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f8e3f-90f5-4606-a10a-b784295d8819 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:19.824100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 323/656] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:19.824627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f8e3f-90f5-4606-a10a-b784295d8819 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.856284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:19.859828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000fdfc-f77b-45b2-8743-ed217384f0ec None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:19.906200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fef12-306c-4ad2-8fad-7bfc932076db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:19.906811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fef12-306c-4ad2-8fad-7bfc932076db None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:19.907087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fef12-306c-4ad2-8fad-7bfc932076db None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:19.907864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fef12-306c-4ad2-8fad-7bfc932076db None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:19.908616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fef12-306c-4ad2-8fad-7bfc932076db None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:19.908921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fef12-306c-4ad2-8fad-7bfc932076db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:19.910395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fef12-306c-4ad2-8fad-7bfc932076db None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:19.931502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 334/657] 10.222.0.23 () {56 vars in 1139 bytes} [Wed Oct 5 20:09:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:19.932427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fef12-306c-4ad2-8fad-7bfc932076db None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:19.951362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f8e3f-90f5-4606-a10a-b784295d8819 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:20.008310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd39ff-eda6-4b3a-8f7e-d68527358e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:20.009348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd39ff-eda6-4b3a-8f7e-d68527358e8c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.009976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd39ff-eda6-4b3a-8f7e-d68527358e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.010417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd39ff-eda6-4b3a-8f7e-d68527358e8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.025717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd39ff-eda6-4b3a-8f7e-d68527358e8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:20.028362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd39ff-eda6-4b3a-8f7e-d68527358e8c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:20.046654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fef12-306c-4ad2-8fad-7bfc932076db None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-75bd39ff-eda6-4b3a-8f7e-d68527358e8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:20.048801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:20.049928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:20.050989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:20.050989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:20.050989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:20.053243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 324/658] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:20.054211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd39ff-eda6-4b3a-8f7e-d68527358e8c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:20.086920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01141d-9b6d-4220-9a16-af5838e662dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:20.087755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01141d-9b6d-4220-9a16-af5838e662dc None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.088142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01141d-9b6d-4220-9a16-af5838e662dc None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.088525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01141d-9b6d-4220-9a16-af5838e662dc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.092107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:20.097819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01141d-9b6d-4220-9a16-af5838e662dc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:20.099769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01141d-9b6d-4220-9a16-af5838e662dc None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-9e01141d-9b6d-4220-9a16-af5838e662dc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:20.113853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:20.114897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 20:09:20.115983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:20.117890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 335/659] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:20.118981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01141d-9b6d-4220-9a16-af5838e662dc None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:20.139860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:20.187873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd39ff-eda6-4b3a-8f7e-d68527358e8c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:20.229739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01141d-9b6d-4220-9a16-af5838e662dc None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:20.230274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efa182-addc-446f-b7d3-7c123ed43e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:20.231047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efa182-addc-446f-b7d3-7c123ed43e88 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.231473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efa182-addc-446f-b7d3-7c123ed43e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.232022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efa182-addc-446f-b7d3-7c123ed43e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.270732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ac94b-b06f-4c9b-ab87-8bf05a611be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:20.271416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ac94b-b06f-4c9b-ab87-8bf05a611be5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.271833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ac94b-b06f-4c9b-ab87-8bf05a611be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.272212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ac94b-b06f-4c9b-ab87-8bf05a611be5 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.272960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ac94b-b06f-4c9b-ab87-8bf05a611be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:20.273603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ac94b-b06f-4c9b-ab87-8bf05a611be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:20.275808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ac94b-b06f-4c9b-ab87-8bf05a611be5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:20.289431 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 336/660] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:20.290533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ac94b-b06f-4c9b-ab87-8bf05a611be5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:20.323685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efa182-addc-446f-b7d3-7c123ed43e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:20.326105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efa182-addc-446f-b7d3-7c123ed43e88 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:20.362807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:20.387086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:20.415176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ac94b-b06f-4c9b-ab87-8bf05a611be5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:20.459411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262dba4-7e1c-4b70-af4c-03e309bf4a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:20.460181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262dba4-7e1c-4b70-af4c-03e309bf4a8c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.460623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262dba4-7e1c-4b70-af4c-03e309bf4a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.461004 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262dba4-7e1c-4b70-af4c-03e309bf4a8c None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.461914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262dba4-7e1c-4b70-af4c-03e309bf4a8c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:20.462433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262dba4-7e1c-4b70-af4c-03e309bf4a8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:20.464660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262dba4-7e1c-4b70-af4c-03e309bf4a8c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:20.467182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 325/661] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:09:20.469044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efa182-addc-446f-b7d3-7c123ed43e88 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:20.484639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 337/662] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 20:09:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:20.486282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262dba4-7e1c-4b70-af4c-03e309bf4a8c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:20.577837 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efa182-addc-446f-b7d3-7c123ed43e88 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:20.597977 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262dba4-7e1c-4b70-af4c-03e309bf4a8c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:20.602542 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:20.611690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c042ad6-8f29-485b-abe3-48853375a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:20.612311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c042ad6-8f29-485b-abe3-48853375a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.612583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c042ad6-8f29-485b-abe3-48853375a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.612850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c042ad6-8f29-485b-abe3-48853375a3b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.619545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c042ad6-8f29-485b-abe3-48853375a3b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:20.620977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c042ad6-8f29-485b-abe3-48853375a3b8 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-2c042ad6-8f29-485b-abe3-48853375a3b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:20.632581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:20.633458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 326/663] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a329b2b-c551-4a2e-9b04-20fae8b2f2f3 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.635147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a329b2b-c551-4a2e-9b04-20fae8b2f2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.635737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a329b2b-c551-4a2e-9b04-20fae8b2f2f3 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.640945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 327/664] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:20.641755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a329b2b-c551-4a2e-9b04-20fae8b2f2f3 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:20.643929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859516-6a02-4fbf-ba5c-27989dd59ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:20.644571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859516-6a02-4fbf-ba5c-27989dd59ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.644852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859516-6a02-4fbf-ba5c-27989dd59ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.645120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da859516-6a02-4fbf-ba5c-27989dd59ed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.647932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:20.656062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da859516-6a02-4fbf-ba5c-27989dd59ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:20.658042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da859516-6a02-4fbf-ba5c-27989dd59ed1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-da859516-6a02-4fbf-ba5c-27989dd59ed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:20.672719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:20.674193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:20.675037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:20.675037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:20.675037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:20.675037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 338/665] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:20.675263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859516-6a02-4fbf-ba5c-27989dd59ed1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:20.763543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a329b2b-c551-4a2e-9b04-20fae8b2f2f3 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:20.781552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da859516-6a02-4fbf-ba5c-27989dd59ed1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:20.803950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fc39e-5c47-4911-8903-abe4e8b704a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:20.804557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fc39e-5c47-4911-8903-abe4e8b704a6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.804832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fc39e-5c47-4911-8903-abe4e8b704a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.805427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fc39e-5c47-4911-8903-abe4e8b704a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.820536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050e61c-c9c5-4bb9-9fde-854d87d4e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:20.821147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050e61c-c9c5-4bb9-9fde-854d87d4e31d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.821426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050e61c-c9c5-4bb9-9fde-854d87d4e31d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.821724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050e61c-c9c5-4bb9-9fde-854d87d4e31d None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.822444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050e61c-c9c5-4bb9-9fde-854d87d4e31d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:20.822726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050e61c-c9c5-4bb9-9fde-854d87d4e31d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:20.824145 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050e61c-c9c5-4bb9-9fde-854d87d4e31d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:20.840759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 339/666] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:20.840759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c30d5-4d57-4587-a20e-727a562a20b1 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:20.840759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c30d5-4d57-4587-a20e-727a562a20b1 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:20.840759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c30d5-4d57-4587-a20e-727a562a20b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:20.857721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-301c30d5-4d57-4587-a20e-727a562a20b1 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:20.859395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-301c30d5-4d57-4587-a20e-727a562a20b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:20.874901 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:20.888769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fc39e-5c47-4911-8903-abe4e8b704a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:20.890386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fc39e-5c47-4911-8903-abe4e8b704a6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:20.912638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:20.936255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 340/667] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:20.937146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c30d5-4d57-4587-a20e-727a562a20b1 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.001490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 328/668] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:09:21.002384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fc39e-5c47-4911-8903-abe4e8b704a6 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.040210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c30d5-4d57-4587-a20e-727a562a20b1 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.076275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb48ff-d157-42d0-a9d1-9160102546ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.076996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb48ff-d157-42d0-a9d1-9160102546ce None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.077273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb48ff-d157-42d0-a9d1-9160102546ce None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.077830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb48ff-d157-42d0-a9d1-9160102546ce None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.078202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb48ff-d157-42d0-a9d1-9160102546ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:21.078502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb48ff-d157-42d0-a9d1-9160102546ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:21.080369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb48ff-d157-42d0-a9d1-9160102546ce None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:21.088243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:21.091826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 341/669] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:21.098322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3b6f6-3f47-468a-868a-24fb84882ebe None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.098834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3b6f6-3f47-468a-868a-24fb84882ebe None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.098962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3b6f6-3f47-468a-868a-24fb84882ebe None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.101556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 342/670] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:21.101556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3b6f6-3f47-468a-868a-24fb84882ebe None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.108589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fc39e-5c47-4911-8903-abe4e8b704a6 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.143344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77555a-1900-476c-aaec-edf23c6593c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.143941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77555a-1900-476c-aaec-edf23c6593c3 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.144212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77555a-1900-476c-aaec-edf23c6593c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.144479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77555a-1900-476c-aaec-edf23c6593c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.145188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77555a-1900-476c-aaec-edf23c6593c3 None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=47bafea2367e4c7992354bb2c23e7d57) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:21.145390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:21.145503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77555a-1900-476c-aaec-edf23c6593c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:21.146900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77555a-1900-476c-aaec-edf23c6593c3 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:21.153620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 329/671] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/role_assignments?user.id=47bafea2367e4c7992354bb2c23e7d57&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:21.154472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77555a-1900-476c-aaec-edf23c6593c3 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.240021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3b6f6-3f47-468a-868a-24fb84882ebe None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.264189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77555a-1900-476c-aaec-edf23c6593c3 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.280308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136f479-96cb-4996-b558-fe8e96c46cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.280954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136f479-96cb-4996-b558-fe8e96c46cdd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.281232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136f479-96cb-4996-b558-fe8e96c46cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.281503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136f479-96cb-4996-b558-fe8e96c46cdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.294849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136f479-96cb-4996-b558-fe8e96c46cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:21.296358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136f479-96cb-4996-b558-fe8e96c46cdd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:21.306271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7907a-9904-4699-8216-9cd627f18698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.307493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7907a-9904-4699-8216-9cd627f18698 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.307956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7907a-9904-4699-8216-9cd627f18698 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.308410 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7907a-9904-4699-8216-9cd627f18698 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-f136f479-96cb-4996-b558-fe8e96c46cdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:21.311932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:21.312916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:21.313867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:21.313867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:21.313867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:21.313867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 343/672] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:21.314633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e3274-393d-4053-a204-b69b8a716fae None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.314867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e3274-393d-4053-a204-b69b8a716fae None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.315135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e3274-393d-4053-a204-b69b8a716fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.319768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7907a-9904-4699-8216-9cd627f18698 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:21.321573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7907a-9904-4699-8216-9cd627f18698 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:21.335688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ff6e3274-393d-4053-a204-b69b8a716fae None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-b7e7907a-9904-4699-8216-9cd627f18698 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:21.336138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:21.337039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:21.338068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:21.338068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:21.338068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:21.338068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:21.338068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:21.338068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:21.338068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:21.338068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:21.338479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ff6e3274-393d-4053-a204-b69b8a716fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:21.338558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 330/673] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:21.338558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7907a-9904-4699-8216-9cd627f18698 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.351541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:21.414450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 344/674] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:21.415596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e3274-393d-4053-a204-b69b8a716fae None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.443524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:21.458466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7907a-9904-4699-8216-9cd627f18698 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.493323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c9e5c-f9ac-4b5e-81b4-406a5c4342e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.493985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c9e5c-f9ac-4b5e-81b4-406a5c4342e1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.494262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c9e5c-f9ac-4b5e-81b4-406a5c4342e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.494534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c9e5c-f9ac-4b5e-81b4-406a5c4342e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.505573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c9e5c-f9ac-4b5e-81b4-406a5c4342e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-581c9e5c-f9ac-4b5e-81b4-406a5c4342e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:21.510772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:21.511644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:21.512838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 331/675] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:21.513490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c9e5c-f9ac-4b5e-81b4-406a5c4342e1 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.542340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e3274-393d-4053-a204-b69b8a716fae None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.581147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aceedd-7744-440e-93fa-8ae06f1ece5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.581799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aceedd-7744-440e-93fa-8ae06f1ece5a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.582184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aceedd-7744-440e-93fa-8ae06f1ece5a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.582461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aceedd-7744-440e-93fa-8ae06f1ece5a None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.583201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aceedd-7744-440e-93fa-8ae06f1ece5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:21.583505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aceedd-7744-440e-93fa-8ae06f1ece5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:21.584863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aceedd-7744-440e-93fa-8ae06f1ece5a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:21.590166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:21.599539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 345/676] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:21.600831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aceedd-7744-440e-93fa-8ae06f1ece5a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.622487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c9e5c-f9ac-4b5e-81b4-406a5c4342e1 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.657295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26612198-4e6e-41ed-909e-d40fbba79396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.657952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26612198-4e6e-41ed-909e-d40fbba79396 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.658231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26612198-4e6e-41ed-909e-d40fbba79396 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.658504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26612198-4e6e-41ed-909e-d40fbba79396 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.659199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26612198-4e6e-41ed-909e-d40fbba79396 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:21.659503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26612198-4e6e-41ed-909e-d40fbba79396 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:21.660929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26612198-4e6e-41ed-909e-d40fbba79396 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:21.678075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 332/677] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:21.678075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26612198-4e6e-41ed-909e-d40fbba79396 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.698348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:21.721908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aceedd-7744-440e-93fa-8ae06f1ece5a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.759898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761fb44-6354-4c4f-ad9b-15a9a374d194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.760512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761fb44-6354-4c4f-ad9b-15a9a374d194 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.760791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761fb44-6354-4c4f-ad9b-15a9a374d194 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.761134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761fb44-6354-4c4f-ad9b-15a9a374d194 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.761825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761fb44-6354-4c4f-ad9b-15a9a374d194 None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:21.762132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761fb44-6354-4c4f-ad9b-15a9a374d194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:21.763489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761fb44-6354-4c4f-ad9b-15a9a374d194 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:21.771969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 346/678] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:21.772768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761fb44-6354-4c4f-ad9b-15a9a374d194 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.784119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26612198-4e6e-41ed-909e-d40fbba79396 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.816755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:21.823370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012a8c-97d9-490b-8405-8d8d9d03fd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.823993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012a8c-97d9-490b-8405-8d8d9d03fd42 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.824274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012a8c-97d9-490b-8405-8d8d9d03fd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.824545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012a8c-97d9-490b-8405-8d8d9d03fd42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.831231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97012a8c-97d9-490b-8405-8d8d9d03fd42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:21.832673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97012a8c-97d9-490b-8405-8d8d9d03fd42 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-97012a8c-97d9-490b-8405-8d8d9d03fd42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:21.844204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:21.850023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 333/679] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:21.850884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012a8c-97d9-490b-8405-8d8d9d03fd42 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.894521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761fb44-6354-4c4f-ad9b-15a9a374d194 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:21.930230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba616b68-75ae-4c9d-9938-08cca2797cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:21.930834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba616b68-75ae-4c9d-9938-08cca2797cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:21.931113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba616b68-75ae-4c9d-9938-08cca2797cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:21.931384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba616b68-75ae-4c9d-9938-08cca2797cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:21.932112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba616b68-75ae-4c9d-9938-08cca2797cf1 None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=8f5a88f263014f15a6f59cfdaa2bb6d8) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:21.932419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba616b68-75ae-4c9d-9938-08cca2797cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:21.933867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba616b68-75ae-4c9d-9938-08cca2797cf1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:21.941586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 347/680] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/role_assignments?user.id=8f5a88f263014f15a6f59cfdaa2bb6d8&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:21.942389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba616b68-75ae-4c9d-9938-08cca2797cf1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:21.957612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:21.972986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012a8c-97d9-490b-8405-8d8d9d03fd42 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.030758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e99c8-4521-4497-b661-fef4ab6ce3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.032970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e99c8-4521-4497-b661-fef4ab6ce3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.033592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e99c8-4521-4497-b661-fef4ab6ce3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.034069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e99c8-4521-4497-b661-fef4ab6ce3b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.049762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e99c8-4521-4497-b661-fef4ab6ce3b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-317e99c8-4521-4497-b661-fef4ab6ce3b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:22.057385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:22.058627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:22.059634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:22.065990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 334/681] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:22.067829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e99c8-4521-4497-b661-fef4ab6ce3b9 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:22.089121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba616b68-75ae-4c9d-9938-08cca2797cf1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.132919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc52810-7d4d-43f1-b98a-8b73ee26d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.133486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc52810-7d4d-43f1-b98a-8b73ee26d35b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.133792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc52810-7d4d-43f1-b98a-8b73ee26d35b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.134067 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc52810-7d4d-43f1-b98a-8b73ee26d35b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.147522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:22.148836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc52810-7d4d-43f1-b98a-8b73ee26d35b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:22.150734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc52810-7d4d-43f1-b98a-8b73ee26d35b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-0bc52810-7d4d-43f1-b98a-8b73ee26d35b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:22.171401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:22.172643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:22.173871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:22.173871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:22.173871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:22.173871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 348/682] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:22.173871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc52810-7d4d-43f1-b98a-8b73ee26d35b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:22.208112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e99c8-4521-4497-b661-fef4ab6ce3b9 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.248707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:22.253434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788df27-54d0-457f-9ea1-a6678ce34bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.254606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788df27-54d0-457f-9ea1-a6678ce34bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.254892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788df27-54d0-457f-9ea1-a6678ce34bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.255316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788df27-54d0-457f-9ea1-a6678ce34bc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.256127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788df27-54d0-457f-9ea1-a6678ce34bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:22.256439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788df27-54d0-457f-9ea1-a6678ce34bc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:22.258159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788df27-54d0-457f-9ea1-a6678ce34bc8 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:22.278932 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 335/683] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:22.279948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788df27-54d0-457f-9ea1-a6678ce34bc8 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:22.299893 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc52810-7d4d-43f1-b98a-8b73ee26d35b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.340242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e3c5-547a-4a64-9486-b7ad108fe3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.341322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e3c5-547a-4a64-9486-b7ad108fe3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.341626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e3c5-547a-4a64-9486-b7ad108fe3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.342165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e3c5-547a-4a64-9486-b7ad108fe3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.342829 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e3c5-547a-4a64-9486-b7ad108fe3d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:22.343238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e3c5-547a-4a64-9486-b7ad108fe3d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:22.344725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e3c5-547a-4a64-9486-b7ad108fe3d5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:22.354961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 349/684] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:22.355796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e3c5-547a-4a64-9486-b7ad108fe3d5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:22.395243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788df27-54d0-457f-9ea1-a6678ce34bc8 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.408829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:22.440477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b76dbf-d557-4d0e-86da-683771c9efc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.441251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b76dbf-d557-4d0e-86da-683771c9efc8 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.441623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b76dbf-d557-4d0e-86da-683771c9efc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.441894 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b76dbf-d557-4d0e-86da-683771c9efc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.442527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b76dbf-d557-4d0e-86da-683771c9efc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:22.442824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b76dbf-d557-4d0e-86da-683771c9efc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:22.444331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b76dbf-d557-4d0e-86da-683771c9efc8 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:22.461605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 336/685] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:22.462477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b76dbf-d557-4d0e-86da-683771c9efc8 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:22.480686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e3c5-547a-4a64-9486-b7ad108fe3d5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.505173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:22.525695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26627484-06e6-4aac-8d3d-db04e845d5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.526962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26627484-06e6-4aac-8d3d-db04e845d5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.527528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26627484-06e6-4aac-8d3d-db04e845d5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.528132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26627484-06e6-4aac-8d3d-db04e845d5a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.529531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26627484-06e6-4aac-8d3d-db04e845d5a7 None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=a6e8bc66986d48aa9dcbc19258ff6e1f) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:22.530162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26627484-06e6-4aac-8d3d-db04e845d5a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:22.532143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26627484-06e6-4aac-8d3d-db04e845d5a7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:22.540677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 350/686] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:22 2022] GET /identity/v3/role_assignments?user.id=a6e8bc66986d48aa9dcbc19258ff6e1f&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:22.542286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26627484-06e6-4aac-8d3d-db04e845d5a7 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:22.582705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b76dbf-d557-4d0e-86da-683771c9efc8 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.658714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc6cd5-018b-4a55-bdd6-6bb784b68190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.659279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc6cd5-018b-4a55-bdd6-6bb784b68190 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.659553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc6cd5-018b-4a55-bdd6-6bb784b68190 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.659820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc6cd5-018b-4a55-bdd6-6bb784b68190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.680870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc6cd5-018b-4a55-bdd6-6bb784b68190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:22.682448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc6cd5-018b-4a55-bdd6-6bb784b68190 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:22.687091 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26627484-06e6-4aac-8d3d-db04e845d5a7 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-8abc6cd5-018b-4a55-bdd6-6bb784b68190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:22.712024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:22.713972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:22.715787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:22.715787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:22.715787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:22.715787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 337/687] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:22.722022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc6cd5-018b-4a55-bdd6-6bb784b68190 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:22.758795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541680ec-fa5d-4e27-892c-3557f2113538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.759481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541680ec-fa5d-4e27-892c-3557f2113538 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.759763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541680ec-fa5d-4e27-892c-3557f2113538 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.760034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541680ec-fa5d-4e27-892c-3557f2113538 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.772853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:22.775930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541680ec-fa5d-4e27-892c-3557f2113538 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:22.779158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541680ec-fa5d-4e27-892c-3557f2113538 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-541680ec-fa5d-4e27-892c-3557f2113538 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:22.797428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:22.798479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:22.799389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 351/688] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:22.804708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541680ec-fa5d-4e27-892c-3557f2113538 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:22.861863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:22.883374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc6cd5-018b-4a55-bdd6-6bb784b68190 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.928210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d6f85-4093-406a-8130-20be2642efad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.928839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d6f85-4093-406a-8130-20be2642efad None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.929123 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d6f85-4093-406a-8130-20be2642efad None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.929499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d6f85-4093-406a-8130-20be2642efad None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.930179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d6f85-4093-406a-8130-20be2642efad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:22.930481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d6f85-4093-406a-8130-20be2642efad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:22.931956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d6f85-4093-406a-8130-20be2642efad None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:22.936065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541680ec-fa5d-4e27-892c-3557f2113538 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:22.946214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 338/689] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:22.951326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d6f85-4093-406a-8130-20be2642efad None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:22.988676 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65cdaa-2c15-4b15-b548-d662ea3ee5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:22.989377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65cdaa-2c15-4b15-b548-d662ea3ee5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:22.989763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65cdaa-2c15-4b15-b548-d662ea3ee5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:22.990097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65cdaa-2c15-4b15-b548-d662ea3ee5b6 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:22.990930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65cdaa-2c15-4b15-b548-d662ea3ee5b6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:22.991236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65cdaa-2c15-4b15-b548-d662ea3ee5b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:22.992660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65cdaa-2c15-4b15-b548-d662ea3ee5b6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:23.011070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 352/690] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 20:09:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:23.016905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65cdaa-2c15-4b15-b548-d662ea3ee5b6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:23.071973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:23.106458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d6f85-4093-406a-8130-20be2642efad None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:23.150959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d2c7-3f1c-4170-bba8-9ebecf019b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:23.151703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d2c7-3f1c-4170-bba8-9ebecf019b5a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:23.151971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d2c7-3f1c-4170-bba8-9ebecf019b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:23.152240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d2c7-3f1c-4170-bba8-9ebecf019b5a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:23.166559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90d2c7-3f1c-4170-bba8-9ebecf019b5a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:23.168285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90d2c7-3f1c-4170-bba8-9ebecf019b5a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:23.172994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65cdaa-2c15-4b15-b548-d662ea3ee5b6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-bb90d2c7-3f1c-4170-bba8-9ebecf019b5a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:23.191786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:23.192752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:23.193764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:23.197386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:23.203373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 339/691] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 20:09:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:23.209748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d272aa5-8cfc-4e14-afd4-32fa01adf59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:23.210405 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d2c7-3f1c-4170-bba8-9ebecf019b5a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:23.210517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d272aa5-8cfc-4e14-afd4-32fa01adf59b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:23.210846 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d272aa5-8cfc-4e14-afd4-32fa01adf59b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:23.211448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d272aa5-8cfc-4e14-afd4-32fa01adf59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:23.322530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d272aa5-8cfc-4e14-afd4-32fa01adf59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:23.323988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d272aa5-8cfc-4e14-afd4-32fa01adf59b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:23.363255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d2c7-3f1c-4170-bba8-9ebecf019b5a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:23.404768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eac99c-0bcc-4529-bba3-2bc2cc9e6535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:23.405539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eac99c-0bcc-4529-bba3-2bc2cc9e6535 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:23.405827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eac99c-0bcc-4529-bba3-2bc2cc9e6535 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:23.406095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eac99c-0bcc-4529-bba3-2bc2cc9e6535 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:23.407570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eac99c-0bcc-4529-bba3-2bc2cc9e6535 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:23.407877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eac99c-0bcc-4529-bba3-2bc2cc9e6535 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:23.409609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eac99c-0bcc-4529-bba3-2bc2cc9e6535 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:23.431061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:23.435029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 340/692] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 20:09:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:23.441579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eac99c-0bcc-4529-bba3-2bc2cc9e6535 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:23.510212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 353/693] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 20:09:23.513731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bbcd5-f0db-4e31-b28d-faf25b4a0ba5 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:23.513731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bbcd5-f0db-4e31-b28d-faf25b4a0ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:23.513731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bbcd5-f0db-4e31-b28d-faf25b4a0ba5 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:23.515131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 354/694] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:09:23.515838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bbcd5-f0db-4e31-b28d-faf25b4a0ba5 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:23.523084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:23.598361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eac99c-0bcc-4529-bba3-2bc2cc9e6535 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:23.633939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bbcd5-f0db-4e31-b28d-faf25b4a0ba5 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:23.638739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109d98-6855-4593-a7aa-7678916c5990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:23.639348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109d98-6855-4593-a7aa-7678916c5990 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:23.639624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109d98-6855-4593-a7aa-7678916c5990 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:23.639894 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab109d98-6855-4593-a7aa-7678916c5990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:23.674581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec5277-443a-4057-83d3-ed55aba4e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:23.675304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec5277-443a-4057-83d3-ed55aba4e82f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:23.675652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec5277-443a-4057-83d3-ed55aba4e82f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:23.675998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec5277-443a-4057-83d3-ed55aba4e82f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:23.686855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec5277-443a-4057-83d3-ed55aba4e82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:23.689293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec5277-443a-4057-83d3-ed55aba4e82f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-acec5277-443a-4057-83d3-ed55aba4e82f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:23.705261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:23.706227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:23.707480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:23.707480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:23.707480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:23.707480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 355/695] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:23.708580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9873822-9219-49a0-8d92-39493e7f2fff None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:23.708685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9873822-9219-49a0-8d92-39493e7f2fff None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:23.709019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9873822-9219-49a0-8d92-39493e7f2fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:23.713615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:23.716407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab109d98-6855-4593-a7aa-7678916c5990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:23.718057 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab109d98-6855-4593-a7aa-7678916c5990 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:23.727753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e9873822-9219-49a0-8d92-39493e7f2fff None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:23.729094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e9873822-9219-49a0-8d92-39493e7f2fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:23.782125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:23.801735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 356/696] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:23.809794 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9873822-9219-49a0-8d92-39493e7f2fff None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:23.841982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 341/697] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 20:09:23.847788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109d98-6855-4593-a7aa-7678916c5990 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:23.961014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9873822-9219-49a0-8d92-39493e7f2fff None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:24.021223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:24.025210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab109d98-6855-4593-a7aa-7678916c5990 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:24.034915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffa9b8-a4b4-4fe4-be98-912a6b124ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:24.035563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffa9b8-a4b4-4fe4-be98-912a6b124ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.035841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffa9b8-a4b4-4fe4-be98-912a6b124ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.036108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffa9b8-a4b4-4fe4-be98-912a6b124ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.036757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffa9b8-a4b4-4fe4-be98-912a6b124ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:24.037054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffa9b8-a4b4-4fe4-be98-912a6b124ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:24.038590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffa9b8-a4b4-4fe4-be98-912a6b124ee6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:24.064615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 357/698] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:24.065878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684f582-d5de-4e3b-a5b1-06a232299a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:24.066759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684f582-d5de-4e3b-a5b1-06a232299a2c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.067221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684f582-d5de-4e3b-a5b1-06a232299a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.067682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684f582-d5de-4e3b-a5b1-06a232299a2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.069850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffa9b8-a4b4-4fe4-be98-912a6b124ee6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:24.079370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684f582-d5de-4e3b-a5b1-06a232299a2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:24.080808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684f582-d5de-4e3b-a5b1-06a232299a2c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-d684f582-d5de-4e3b-a5b1-06a232299a2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:24.095573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:24.096561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:24.097415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:24.097415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:24.097415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:24.097415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 342/699] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:24.102465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684f582-d5de-4e3b-a5b1-06a232299a2c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:24.126905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:24.215836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffa9b8-a4b4-4fe4-be98-912a6b124ee6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:24.255739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684f582-d5de-4e3b-a5b1-06a232299a2c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:24.258528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96135181-e411-4b43-8c7f-a0c069228865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:24.259446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96135181-e411-4b43-8c7f-a0c069228865 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.259917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96135181-e411-4b43-8c7f-a0c069228865 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.260381 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96135181-e411-4b43-8c7f-a0c069228865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.267736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96135181-e411-4b43-8c7f-a0c069228865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:24.269692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96135181-e411-4b43-8c7f-a0c069228865 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-96135181-e411-4b43-8c7f-a0c069228865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:24.283831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:24.284776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:24.285736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:24.285736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:24.285736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:24.285736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:24.285736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:24.285736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:24.285736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:24.285736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:24.285736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 358/700] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:24.291689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96135181-e411-4b43-8c7f-a0c069228865 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:24.304082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea76105-8e1e-4367-a87a-6e108a8bd309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:24.305635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea76105-8e1e-4367-a87a-6e108a8bd309 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.306425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea76105-8e1e-4367-a87a-6e108a8bd309 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.306908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea76105-8e1e-4367-a87a-6e108a8bd309 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.308073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea76105-8e1e-4367-a87a-6e108a8bd309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:24.308596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea76105-8e1e-4367-a87a-6e108a8bd309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:24.311162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea76105-8e1e-4367-a87a-6e108a8bd309 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:24.334852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 343/701] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:24.336365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea76105-8e1e-4367-a87a-6e108a8bd309 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:24.355002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:24.419521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96135181-e411-4b43-8c7f-a0c069228865 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:24.423848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:24.456248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4032a70-fe53-4fa0-b3bb-c7f7ba3877c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:24.456986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4032a70-fe53-4fa0-b3bb-c7f7ba3877c6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.457259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4032a70-fe53-4fa0-b3bb-c7f7ba3877c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.457676 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4032a70-fe53-4fa0-b3bb-c7f7ba3877c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.460417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea76105-8e1e-4367-a87a-6e108a8bd309 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:24.468538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4032a70-fe53-4fa0-b3bb-c7f7ba3877c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:24.470661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4032a70-fe53-4fa0-b3bb-c7f7ba3877c6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-a4032a70-fe53-4fa0-b3bb-c7f7ba3877c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:24.486088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:24.487200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:24.488069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:24.488069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:24.488069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:24.488069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 359/702] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:24.488339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4032a70-fe53-4fa0-b3bb-c7f7ba3877c6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:24.500437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41aacfe-adc5-44bb-8b93-c698458620cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:24.501105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41aacfe-adc5-44bb-8b93-c698458620cd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.501383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41aacfe-adc5-44bb-8b93-c698458620cd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.501696 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41aacfe-adc5-44bb-8b93-c698458620cd None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.502382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41aacfe-adc5-44bb-8b93-c698458620cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:24.502680 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41aacfe-adc5-44bb-8b93-c698458620cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:24.504088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41aacfe-adc5-44bb-8b93-c698458620cd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:24.515362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 344/703] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:24.516554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af600b80-31ac-4fdc-a197-d2dc365010d8 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.516838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af600b80-31ac-4fdc-a197-d2dc365010d8 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.517104 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af600b80-31ac-4fdc-a197-d2dc365010d8 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.518482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 345/704] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:24.522983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af600b80-31ac-4fdc-a197-d2dc365010d8 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:24.595396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:24.599465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4032a70-fe53-4fa0-b3bb-c7f7ba3877c6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:24.627713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af600b80-31ac-4fdc-a197-d2dc365010d8 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:24.632833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb2ce5-4b92-4b64-ac0a-1172f73ec4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:24.633456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb2ce5-4b92-4b64-ac0a-1172f73ec4ce None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.633759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb2ce5-4b92-4b64-ac0a-1172f73ec4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.634031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb2ce5-4b92-4b64-ac0a-1172f73ec4ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.640646 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb2ce5-4b92-4b64-ac0a-1172f73ec4ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:24.642177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb2ce5-4b92-4b64-ac0a-1172f73ec4ce None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-e1bb2ce5-4b92-4b64-ac0a-1172f73ec4ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:24.654374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:24.655274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:24.656151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:24.656151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:24.656151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:24.656151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:24.656151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:24.656151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:24.656151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:24.656151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:24.656151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 360/705] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:24.656641 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb2ce5-4b92-4b64-ac0a-1172f73ec4ce None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:24.671239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639d310-10bd-4b04-ab51-1ff068afb821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:24.671845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639d310-10bd-4b04-ab51-1ff068afb821 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.672119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639d310-10bd-4b04-ab51-1ff068afb821 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.672389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639d310-10bd-4b04-ab51-1ff068afb821 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.673698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639d310-10bd-4b04-ab51-1ff068afb821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:24.674000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639d310-10bd-4b04-ab51-1ff068afb821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:24.675357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639d310-10bd-4b04-ab51-1ff068afb821 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:24.675886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:24.686999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 346/706] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:24.688187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bae050-21f2-4abe-9bc6-fc318b7b0b25 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.688523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bae050-21f2-4abe-9bc6-fc318b7b0b25 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.688799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bae050-21f2-4abe-9bc6-fc318b7b0b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.708265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c0bae050-21f2-4abe-9bc6-fc318b7b0b25 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:24.709446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c0bae050-21f2-4abe-9bc6-fc318b7b0b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:24.767642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb2ce5-4b92-4b64-ac0a-1172f73ec4ce None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:24.789668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 347/707] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:24.790604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bae050-21f2-4abe-9bc6-fc318b7b0b25 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:24.808632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a6b28-0905-4cd8-9d83-ab351e1734cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:24.809664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a6b28-0905-4cd8-9d83-ab351e1734cf None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:24.810155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a6b28-0905-4cd8-9d83-ab351e1734cf None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:24.810768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a6b28-0905-4cd8-9d83-ab351e1734cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:24.811744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a6b28-0905-4cd8-9d83-ab351e1734cf None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=f8dec1dbdcd94316aad25c93c313a5a5) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:24.812244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a6b28-0905-4cd8-9d83-ab351e1734cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:24.814920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a6b28-0905-4cd8-9d83-ab351e1734cf None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:24.821973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 361/708] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:24 2022] GET /identity/v3/role_assignments?user.id=f8dec1dbdcd94316aad25c93c313a5a5&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:24.823105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a6b28-0905-4cd8-9d83-ab351e1734cf None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:24.917396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:25.012075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:25.012591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bae050-21f2-4abe-9bc6-fc318b7b0b25 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:25.044126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a6b28-0905-4cd8-9d83-ab351e1734cf None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:25.060012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bca76-c5e7-4203-b19c-625d90ad82d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:25.060623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bca76-c5e7-4203-b19c-625d90ad82d6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:25.060884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bca76-c5e7-4203-b19c-625d90ad82d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:25.061227 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bca76-c5e7-4203-b19c-625d90ad82d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:25.062241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bca76-c5e7-4203-b19c-625d90ad82d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:25.062241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bca76-c5e7-4203-b19c-625d90ad82d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:25.063679 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bca76-c5e7-4203-b19c-625d90ad82d6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:25.079462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 348/709] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:25.080373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bca76-c5e7-4203-b19c-625d90ad82d6 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:25.087136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a9e23-8585-4eb9-b3bb-4c626113422a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:25.087747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a9e23-8585-4eb9-b3bb-4c626113422a None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:25.088020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a9e23-8585-4eb9-b3bb-4c626113422a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:25.088287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a9e23-8585-4eb9-b3bb-4c626113422a None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:25.088923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a9e23-8585-4eb9-b3bb-4c626113422a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:25.090498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a9e23-8585-4eb9-b3bb-4c626113422a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:25.117665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 362/710] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:09:24 2022] POST /identity/v3/users => generated 254 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:25.118893 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a9e23-8585-4eb9-b3bb-4c626113422a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:25.208851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bca76-c5e7-4203-b19c-625d90ad82d6 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:25.238077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:25.254493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b43b5-eddd-4109-ab27-3a459a23ff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:25.255055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b43b5-eddd-4109-ab27-3a459a23ff4f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:25.255336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b43b5-eddd-4109-ab27-3a459a23ff4f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:25.255603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b43b5-eddd-4109-ab27-3a459a23ff4f None admin] PATH_INFO: `/v3/services/network` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:25.256195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b43b5-eddd-4109-ab27-3a459a23ff4f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:25.257622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b43b5-eddd-4109-ab27-3a459a23ff4f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:25.258427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a9e23-8585-4eb9-b3bb-4c626113422a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-e55b43b5-eddd-4109-ab27-3a459a23ff4f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:25.270829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:25.272096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:25.273060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 349/711] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:25 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:25.278534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b43b5-eddd-4109-ab27-3a459a23ff4f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:25.302755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203076c1-adcb-496b-bc38-d92bff24b8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:25.304126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203076c1-adcb-496b-bc38-d92bff24b8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:25.304550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203076c1-adcb-496b-bc38-d92bff24b8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:25.304970 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203076c1-adcb-496b-bc38-d92bff24b8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:25.305955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203076c1-adcb-496b-bc38-d92bff24b8f1 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=286f4e203e7a4a55bca0a58d061af2bd) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:25.306437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203076c1-adcb-496b-bc38-d92bff24b8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:25.308760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203076c1-adcb-496b-bc38-d92bff24b8f1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:25.314396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:25.319516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 363/712] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:25 2022] GET /identity/v3/role_assignments?user.id=286f4e203e7a4a55bca0a58d061af2bd&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=36331ec562ff42f883fc883363bc4c8d => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:25.323316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ec7aa-1f88-40c0-8d14-c20185cbfde6 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:25.323316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ec7aa-1f88-40c0-8d14-c20185cbfde6 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:25.323316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ec7aa-1f88-40c0-8d14-c20185cbfde6 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:25.325313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 364/713] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:25.331913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb43af-d28c-4220-8910-519aa5d478ef None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:25.332521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb43af-d28c-4220-8910-519aa5d478ef None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:25.332627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb43af-d28c-4220-8910-519aa5d478ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:25.353134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ffbb43af-d28c-4220-8910-519aa5d478ef None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:25.354734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ffbb43af-d28c-4220-8910-519aa5d478ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:25.421139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b43b5-eddd-4109-ab27-3a459a23ff4f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:25.457406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 365/714] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 20:09:25.470454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d45f8-8f6e-4bfe-9056-789c4b069624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:25.470996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d45f8-8f6e-4bfe-9056-789c4b069624 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:25.471269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d45f8-8f6e-4bfe-9056-789c4b069624 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:25.471536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d45f8-8f6e-4bfe-9056-789c4b069624 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:25.472168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d45f8-8f6e-4bfe-9056-789c4b069624 None admin] RBAC: Adding query filter params (name=network) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:25.472467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d45f8-8f6e-4bfe-9056-789c4b069624 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:25.473878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d45f8-8f6e-4bfe-9056-789c4b069624 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:25.479480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 350/715] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:09:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:25.484831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb43af-d28c-4220-8910-519aa5d478ef None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:25.608515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb43af-d28c-4220-8910-519aa5d478ef None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:25.639588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:25.646821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e505d-95a4-458f-96d4-2af74c90ee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:25.647443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e505d-95a4-458f-96d4-2af74c90ee3c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:25.647764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e505d-95a4-458f-96d4-2af74c90ee3c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:25.648042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e505d-95a4-458f-96d4-2af74c90ee3c None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:25.648876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e505d-95a4-458f-96d4-2af74c90ee3c None admin] RBAC: Adding query filter params (type=network) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:25.649346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e505d-95a4-458f-96d4-2af74c90ee3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:25.651317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e505d-95a4-458f-96d4-2af74c90ee3c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:25.657903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 366/716] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:09:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:25.761082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d45f8-8f6e-4bfe-9056-789c4b069624 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:25.833324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:25.909276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d45f8-8f6e-4bfe-9056-789c4b069624 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:25.943472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5bc48-a796-41a3-a886-9be1f134582d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:25.944145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5bc48-a796-41a3-a886-9be1f134582d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:25.944420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5bc48-a796-41a3-a886-9be1f134582d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:25.944686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5bc48-a796-41a3-a886-9be1f134582d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:25.945262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5bc48-a796-41a3-a886-9be1f134582d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:25.946661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5bc48-a796-41a3-a886-9be1f134582d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-f4b5bc48-a796-41a3-a886-9be1f134582d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:25.955601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:25.956623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 20:09:25.957424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:25.961540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 351/717] 10.222.0.23 () {56 vars in 1129 bytes} [Wed Oct 5 20:09:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:25.962903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e505d-95a4-458f-96d4-2af74c90ee3c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:26.070392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e505d-95a4-458f-96d4-2af74c90ee3c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:26.114225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7f754-0755-4e32-a6b9-db03c9ee3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:26.115272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7f754-0755-4e32-a6b9-db03c9ee3b92 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:26.115822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7f754-0755-4e32-a6b9-db03c9ee3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:26.116298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7f754-0755-4e32-a6b9-db03c9ee3b92 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:26.117179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7f754-0755-4e32-a6b9-db03c9ee3b92 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:26.117706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7f754-0755-4e32-a6b9-db03c9ee3b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:26.119292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7f754-0755-4e32-a6b9-db03c9ee3b92 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:26.124359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 367/718] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 20:09:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:26.130063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5bc48-a796-41a3-a886-9be1f134582d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:26.285400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5bc48-a796-41a3-a886-9be1f134582d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:26.304524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:26.330731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba101ec-5774-48c6-8a36-9b0d480d8a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:26.331270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba101ec-5774-48c6-8a36-9b0d480d8a02 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:26.331546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba101ec-5774-48c6-8a36-9b0d480d8a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:26.331815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba101ec-5774-48c6-8a36-9b0d480d8a02 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:26.332460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba101ec-5774-48c6-8a36-9b0d480d8a02 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:26.332756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba101ec-5774-48c6-8a36-9b0d480d8a02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:26.334209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba101ec-5774-48c6-8a36-9b0d480d8a02 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:26.347421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 352/719] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 20:09:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:27.640674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c87d5-ac2d-4fd9-bad2-741df8a0fe81 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:27.641190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c87d5-ac2d-4fd9-bad2-741df8a0fe81 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:27.641190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c87d5-ac2d-4fd9-bad2-741df8a0fe81 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:27.643476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 368/720] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:27.648396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3afda2-c969-4794-8363-02e369dc55c4 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:27.648782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3afda2-c969-4794-8363-02e369dc55c4 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:27.648946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3afda2-c969-4794-8363-02e369dc55c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:27.673619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1f3afda2-c969-4794-8363-02e369dc55c4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:27.674816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1f3afda2-c969-4794-8363-02e369dc55c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:27.771670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 353/721] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:27.815505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf563f54-7534-4891-b700-279ec143701c None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:27.816108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf563f54-7534-4891-b700-279ec143701c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:27.816615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf563f54-7534-4891-b700-279ec143701c None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:27.818364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 369/722] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:27.823740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fae85-f910-4f43-88fd-84b339672c90 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:27.824028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fae85-f910-4f43-88fd-84b339672c90 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:27.824305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fae85-f910-4f43-88fd-84b339672c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:27.847784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-938fae85-f910-4f43-88fd-84b339672c90 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:27.848956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-938fae85-f910-4f43-88fd-84b339672c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:27.948151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 354/723] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:28.062040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf563f54-7534-4891-b700-279ec143701c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:28.127562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:28.182721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf563f54-7534-4891-b700-279ec143701c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:28.229717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d479c0-e4f9-4dae-bffa-f1a0221154d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:28.230664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d479c0-e4f9-4dae-bffa-f1a0221154d3 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.231136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d479c0-e4f9-4dae-bffa-f1a0221154d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.231609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d479c0-e4f9-4dae-bffa-f1a0221154d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.250309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d479c0-e4f9-4dae-bffa-f1a0221154d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-e3d479c0-e4f9-4dae-bffa-f1a0221154d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:28.255757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:28.256653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:28.257463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:28.259703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 370/724] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:28.265751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fae85-f910-4f43-88fd-84b339672c90 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:28.305212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d479c0-e4f9-4dae-bffa-f1a0221154d3 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:28.483077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:28.488667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d479c0-e4f9-4dae-bffa-f1a0221154d3 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:28.518985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:28.556983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fae85-f910-4f43-88fd-84b339672c90 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:28.558043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698818d-e36b-4ca0-9b9a-0b811c656fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:28.559201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698818d-e36b-4ca0-9b9a-0b811c656fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.559473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698818d-e36b-4ca0-9b9a-0b811c656fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.560377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698818d-e36b-4ca0-9b9a-0b811c656fa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.575854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698818d-e36b-4ca0-9b9a-0b811c656fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-0698818d-e36b-4ca0-9b9a-0b811c656fa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:28.584530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:28.585497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:28.586701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 371/725] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:28.587533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43420fc4-161b-47ac-8bec-a57c6efaaedd None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.587533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43420fc4-161b-47ac-8bec-a57c6efaaedd None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.587683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43420fc4-161b-47ac-8bec-a57c6efaaedd None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.593342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 372/726] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:28.593342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43420fc4-161b-47ac-8bec-a57c6efaaedd None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:28.604933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8d580-7583-4116-a4a0-acec6469b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:28.605811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8d580-7583-4116-a4a0-acec6469b8dc None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.606255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8d580-7583-4116-a4a0-acec6469b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.606680 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8d580-7583-4116-a4a0-acec6469b8dc None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.607478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8d580-7583-4116-a4a0-acec6469b8dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:28.607915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8d580-7583-4116-a4a0-acec6469b8dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:28.609408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8d580-7583-4116-a4a0-acec6469b8dc None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:28.619029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 355/727] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:28.620758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0cb68-436d-4760-91e0-45b78818cac0 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.621191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0cb68-436d-4760-91e0-45b78818cac0 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.621639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0cb68-436d-4760-91e0-45b78818cac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.641947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-71a0cb68-436d-4760-91e0-45b78818cac0 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:28.643396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-71a0cb68-436d-4760-91e0-45b78818cac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:28.721614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 356/728] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:28.723252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0cb68-436d-4760-91e0-45b78818cac0 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:28.733450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43420fc4-161b-47ac-8bec-a57c6efaaedd None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:28.769229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:28.786906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8f6c8-e148-4f31-b70e-7ae5bc22cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:28.787856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8f6c8-e148-4f31-b70e-7ae5bc22cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.788134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8f6c8-e148-4f31-b70e-7ae5bc22cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.788423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8f6c8-e148-4f31-b70e-7ae5bc22cbf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.789072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8f6c8-e148-4f31-b70e-7ae5bc22cbf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:28.789426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8f6c8-e148-4f31-b70e-7ae5bc22cbf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:28.791645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8f6c8-e148-4f31-b70e-7ae5bc22cbf0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:28.803950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 373/729] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:28.805828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618db7c-ecfc-416f-8998-6f579ae62c29 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.806228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618db7c-ecfc-416f-8998-6f579ae62c29 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.806623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618db7c-ecfc-416f-8998-6f579ae62c29 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.808476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 374/730] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:28.809186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618db7c-ecfc-416f-8998-6f579ae62c29 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:28.855385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0cb68-436d-4760-91e0-45b78818cac0 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:28.871861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:28.895873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bf4ae-a39e-4553-8dc3-80b5b09c64f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:28.897092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bf4ae-a39e-4553-8dc3-80b5b09c64f7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.897598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bf4ae-a39e-4553-8dc3-80b5b09c64f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.898134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bf4ae-a39e-4553-8dc3-80b5b09c64f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.909925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bf4ae-a39e-4553-8dc3-80b5b09c64f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:28.911627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bf4ae-a39e-4553-8dc3-80b5b09c64f7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-802bf4ae-a39e-4553-8dc3-80b5b09c64f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:28.927569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:28.928501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:28.929349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:28.929349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:28.929349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:28.931559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 357/731] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:28.933138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff79226-fe8a-4b53-bff6-15c9c22eff72 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.933792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff79226-fe8a-4b53-bff6-15c9c22eff72 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.934253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff79226-fe8a-4b53-bff6-15c9c22eff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.944175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618db7c-ecfc-416f-8998-6f579ae62c29 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:28.954984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6ff79226-fe8a-4b53-bff6-15c9c22eff72 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:28.956152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6ff79226-fe8a-4b53-bff6-15c9c22eff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:28.983284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbdefa-de03-4638-b8f5-25b89b20ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:28.983964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbdefa-de03-4638-b8f5-25b89b20ce55 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:28.984185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbdefa-de03-4638-b8f5-25b89b20ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:28.984454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbdefa-de03-4638-b8f5-25b89b20ce55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:28.994901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbdefa-de03-4638-b8f5-25b89b20ce55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:28.996319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbdefa-de03-4638-b8f5-25b89b20ce55 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-a7cbdefa-de03-4638-b8f5-25b89b20ce55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:29.011828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:29.012753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:29.013698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:29.013698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:29.013698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:29.013698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 375/732] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:29.014565 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbdefa-de03-4638-b8f5-25b89b20ce55 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.024655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 358/733] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:29.025928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f08d0-cf7b-4147-b55b-104c6910bd3c None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.026208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f08d0-cf7b-4147-b55b-104c6910bd3c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.026474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f08d0-cf7b-4147-b55b-104c6910bd3c None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.029561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 359/734] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:29.029561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f08d0-cf7b-4147-b55b-104c6910bd3c None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.046551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:29.110568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:29.120886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbdefa-de03-4638-b8f5-25b89b20ce55 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.141121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f08d0-cf7b-4147-b55b-104c6910bd3c None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.158260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748afd51-740d-4521-9a20-03e38ddfe909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.159147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748afd51-740d-4521-9a20-03e38ddfe909 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.159431 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748afd51-740d-4521-9a20-03e38ddfe909 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.159706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748afd51-740d-4521-9a20-03e38ddfe909 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.160340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748afd51-740d-4521-9a20-03e38ddfe909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:29.160644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748afd51-740d-4521-9a20-03e38ddfe909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:29.162609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748afd51-740d-4521-9a20-03e38ddfe909 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.174959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 376/735] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:29.175283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaa7ed-4fd3-4eab-b859-4c3c956d2608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.175965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaa7ed-4fd3-4eab-b859-4c3c956d2608 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.176243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaa7ed-4fd3-4eab-b859-4c3c956d2608 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.176511 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaa7ed-4fd3-4eab-b859-4c3c956d2608 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.179940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748afd51-740d-4521-9a20-03e38ddfe909 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.186882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebaa7ed-4fd3-4eab-b859-4c3c956d2608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-cebaa7ed-4fd3-4eab-b859-4c3c956d2608 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:29.192423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:29.193354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:29.194211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 360/736] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:29.194897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab2d09-31a5-4c4e-b4e0-1d820873a607 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.194897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab2d09-31a5-4c4e-b4e0-1d820873a607 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.195797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab2d09-31a5-4c4e-b4e0-1d820873a607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.213473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-afab2d09-31a5-4c4e-b4e0-1d820873a607 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:29.214685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-afab2d09-31a5-4c4e-b4e0-1d820873a607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:29.284381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 361/737] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 20:09:29.285287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab2d09-31a5-4c4e-b4e0-1d820873a607 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.287494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:29.299008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748afd51-740d-4521-9a20-03e38ddfe909 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.334759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda633c-4f71-4c34-9380-4cf771acdade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.335454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda633c-4f71-4c34-9380-4cf771acdade None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.335728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda633c-4f71-4c34-9380-4cf771acdade None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.335993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda633c-4f71-4c34-9380-4cf771acdade None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.336732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda633c-4f71-4c34-9380-4cf771acdade None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:29.337031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda633c-4f71-4c34-9380-4cf771acdade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:29.338570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda633c-4f71-4c34-9380-4cf771acdade None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.350766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 377/738] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:29.354410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda633c-4f71-4c34-9380-4cf771acdade None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.370987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:29.390444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab2d09-31a5-4c4e-b4e0-1d820873a607 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.424222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f72165-3044-4f6b-90f5-4b3587abb60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.424861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f72165-3044-4f6b-90f5-4b3587abb60d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.425139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f72165-3044-4f6b-90f5-4b3587abb60d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.425402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f72165-3044-4f6b-90f5-4b3587abb60d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.432365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f72165-3044-4f6b-90f5-4b3587abb60d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:29.433842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f72165-3044-4f6b-90f5-4b3587abb60d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-23f72165-3044-4f6b-90f5-4b3587abb60d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:29.445967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:29.446859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 362/739] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:29.447774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f72165-3044-4f6b-90f5-4b3587abb60d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.471621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda633c-4f71-4c34-9380-4cf771acdade None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.506437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7fe1-1e29-42cb-b715-67a728c3571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.507121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7fe1-1e29-42cb-b715-67a728c3571c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.507392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7fe1-1e29-42cb-b715-67a728c3571c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.507658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7fe1-1e29-42cb-b715-67a728c3571c None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.508308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b7fe1-1e29-42cb-b715-67a728c3571c None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:29.508607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b7fe1-1e29-42cb-b715-67a728c3571c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:29.510008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b7fe1-1e29-42cb-b715-67a728c3571c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.510743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:29.523751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 378/740] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:29.523751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7fe1-1e29-42cb-b715-67a728c3571c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.557845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f72165-3044-4f6b-90f5-4b3587abb60d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.590900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5d399-67f3-408e-8be3-0f3ad56f38b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.591510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5d399-67f3-408e-8be3-0f3ad56f38b5 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.591786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5d399-67f3-408e-8be3-0f3ad56f38b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.592058 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5d399-67f3-408e-8be3-0f3ad56f38b5 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.592703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5d399-67f3-408e-8be3-0f3ad56f38b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:29.594109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5d399-67f3-408e-8be3-0f3ad56f38b5 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.602036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:29.610697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 363/741] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:09:29 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:29.611519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5d399-67f3-408e-8be3-0f3ad56f38b5 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.630275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7fe1-1e29-42cb-b715-67a728c3571c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.682522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f60c2-f45b-490f-86ab-981d7fe73d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.683165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f60c2-f45b-490f-86ab-981d7fe73d48 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.683445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f60c2-f45b-490f-86ab-981d7fe73d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.683716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f60c2-f45b-490f-86ab-981d7fe73d48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.692617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f60c2-f45b-490f-86ab-981d7fe73d48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:29.694136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f60c2-f45b-490f-86ab-981d7fe73d48 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-e53f60c2-f45b-490f-86ab-981d7fe73d48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:29.706999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:29.708193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:29.709302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:29.712971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 379/742] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:29.713957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f60c2-f45b-490f-86ab-981d7fe73d48 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.732906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5d399-67f3-408e-8be3-0f3ad56f38b5 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.760016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:29.774820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a19a17-8cf0-41a1-899a-b3403efcb7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.775620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a19a17-8cf0-41a1-899a-b3403efcb7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.776004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a19a17-8cf0-41a1-899a-b3403efcb7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.776284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a19a17-8cf0-41a1-899a-b3403efcb7a8 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.777015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a19a17-8cf0-41a1-899a-b3403efcb7a8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:29.777313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a19a17-8cf0-41a1-899a-b3403efcb7a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:29.778786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a19a17-8cf0-41a1-899a-b3403efcb7a8 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.799751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 364/743] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:29.800272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a19a17-8cf0-41a1-899a-b3403efcb7a8 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.833369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f60c2-f45b-490f-86ab-981d7fe73d48 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.869455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:29.880549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc8344-545c-4aba-b517-75d2fdbcc1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.881182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc8344-545c-4aba-b517-75d2fdbcc1ee None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.881768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc8344-545c-4aba-b517-75d2fdbcc1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.881768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc8344-545c-4aba-b517-75d2fdbcc1ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.893667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc8344-545c-4aba-b517-75d2fdbcc1ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:29.895501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc8344-545c-4aba-b517-75d2fdbcc1ee None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.904303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a19a17-8cf0-41a1-899a-b3403efcb7a8 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-00fc8344-545c-4aba-b517-75d2fdbcc1ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:29.911146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:29.912275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:29.913339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:29.913339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:29.913339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:29.916738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 380/744] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:29.917469 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc8344-545c-4aba-b517-75d2fdbcc1ee None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:29.942385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e2c02-d59b-4934-8258-fe96901b04c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:29.943072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e2c02-d59b-4934-8258-fe96901b04c4 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:29.943413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e2c02-d59b-4934-8258-fe96901b04c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:29.943747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e2c02-d59b-4934-8258-fe96901b04c4 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:29.944419 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e2c02-d59b-4934-8258-fe96901b04c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:29.945787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e2c02-d59b-4934-8258-fe96901b04c4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:29.961735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 365/745] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:09:29 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:29.963084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e2c02-d59b-4934-8258-fe96901b04c4 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:30.027947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc8344-545c-4aba-b517-75d2fdbcc1ee None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:30.030994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:30.074037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71cdc7-6013-45f9-a4d8-831b567270f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:30.075031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71cdc7-6013-45f9-a4d8-831b567270f9 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:30.075508 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71cdc7-6013-45f9-a4d8-831b567270f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:30.076021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71cdc7-6013-45f9-a4d8-831b567270f9 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:30.076871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71cdc7-6013-45f9-a4d8-831b567270f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:30.077272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71cdc7-6013-45f9-a4d8-831b567270f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:30.079459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71cdc7-6013-45f9-a4d8-831b567270f9 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:30.105574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 381/746] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:30.105574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71cdc7-6013-45f9-a4d8-831b567270f9 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:30.121308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e2c02-d59b-4934-8258-fe96901b04c4 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:30.147831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:30.178353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecc0f7-cbf2-40cd-b931-78f02f643e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:30.178353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecc0f7-cbf2-40cd-b931-78f02f643e93 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:30.178625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecc0f7-cbf2-40cd-b931-78f02f643e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:30.178798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecc0f7-cbf2-40cd-b931-78f02f643e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:30.235681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71cdc7-6013-45f9-a4d8-831b567270f9 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:30.262154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecc0f7-cbf2-40cd-b931-78f02f643e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:30.264539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecc0f7-cbf2-40cd-b931-78f02f643e93 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:30.282962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8280da-f578-4409-9993-ec40096e399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:30.283697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8280da-f578-4409-9993-ec40096e399a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:30.284068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8280da-f578-4409-9993-ec40096e399a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:30.284430 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8280da-f578-4409-9993-ec40096e399a None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:30.285366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8280da-f578-4409-9993-ec40096e399a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:30.285709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8280da-f578-4409-9993-ec40096e399a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:30.287279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8280da-f578-4409-9993-ec40096e399a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:30.306378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 382/747] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:30.307883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8280da-f578-4409-9993-ec40096e399a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:30.317308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:30.405178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 366/748] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:09:30.405178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecc0f7-cbf2-40cd-b931-78f02f643e93 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:30.421984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:30.429582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8280da-f578-4409-9993-ec40096e399a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:30.466017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5c736-6c9a-4f55-8237-91b59add4d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:30.467029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5c736-6c9a-4f55-8237-91b59add4d4e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:30.467497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5c736-6c9a-4f55-8237-91b59add4d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:30.467998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5c736-6c9a-4f55-8237-91b59add4d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:30.479690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5c736-6c9a-4f55-8237-91b59add4d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:30.481412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5c736-6c9a-4f55-8237-91b59add4d4e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-cce5c736-6c9a-4f55-8237-91b59add4d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:30.496715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:30.497701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:30.498571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:30.498571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:30.498571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:30.501371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 383/749] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:30.502543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5c736-6c9a-4f55-8237-91b59add4d4e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:30.528590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecc0f7-cbf2-40cd-b931-78f02f643e93 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:30.569143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b138e2-2f07-4147-8c03-415d6ed70668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:30.570230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b138e2-2f07-4147-8c03-415d6ed70668 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:30.570817 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b138e2-2f07-4147-8c03-415d6ed70668 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:30.571279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b138e2-2f07-4147-8c03-415d6ed70668 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:30.579643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b138e2-2f07-4147-8c03-415d6ed70668 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:30.581347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b138e2-2f07-4147-8c03-415d6ed70668 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:30.584273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-26b138e2-2f07-4147-8c03-415d6ed70668 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:30.595868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:30.596795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:30.597730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:30.600101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 367/750] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:09:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:30.605490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b138e2-2f07-4147-8c03-415d6ed70668 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:30.622736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5c736-6c9a-4f55-8237-91b59add4d4e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:30.681561 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd0495-b1e8-4bea-acb1-4707da68a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:30.682117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd0495-b1e8-4bea-acb1-4707da68a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:30.682395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd0495-b1e8-4bea-acb1-4707da68a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:30.682667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd0495-b1e8-4bea-acb1-4707da68a5e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:30.697175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd0495-b1e8-4bea-acb1-4707da68a5e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:30.698665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd0495-b1e8-4bea-acb1-4707da68a5e9 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:30.710399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-5dfd0495-b1e8-4bea-acb1-4707da68a5e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:30.719185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:30.720170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:30.721012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:30.721012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:30.721012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:30.728179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 384/751] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:30.728179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd0495-b1e8-4bea-acb1-4707da68a5e9 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:30.733092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b138e2-2f07-4147-8c03-415d6ed70668 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:30.784818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74678aef-5870-4b6e-a57e-f8eec0ca384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:30.785604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678aef-5870-4b6e-a57e-f8eec0ca384f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:30.786258 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678aef-5870-4b6e-a57e-f8eec0ca384f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:30.786630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678aef-5870-4b6e-a57e-f8eec0ca384f None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:30.788308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74678aef-5870-4b6e-a57e-f8eec0ca384f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:30.788732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74678aef-5870-4b6e-a57e-f8eec0ca384f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:30.791054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74678aef-5870-4b6e-a57e-f8eec0ca384f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:30.807377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 368/752] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:30.809501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74678aef-5870-4b6e-a57e-f8eec0ca384f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:30.862797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd0495-b1e8-4bea-acb1-4707da68a5e9 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:30.887291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:30.910417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748da66-63c9-4d41-95fe-5ae5f7ff8855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:30.911269 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748da66-63c9-4d41-95fe-5ae5f7ff8855 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:30.911561 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748da66-63c9-4d41-95fe-5ae5f7ff8855 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:30.911991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748da66-63c9-4d41-95fe-5ae5f7ff8855 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:30.913240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748da66-63c9-4d41-95fe-5ae5f7ff8855 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:30.913613 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748da66-63c9-4d41-95fe-5ae5f7ff8855 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:30.915422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748da66-63c9-4d41-95fe-5ae5f7ff8855 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:30.934738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74678aef-5870-4b6e-a57e-f8eec0ca384f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:30.945668 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 385/753] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 20:09:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:30.946843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748da66-63c9-4d41-95fe-5ae5f7ff8855 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:30.975401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b1dd9-db4c-495d-9e99-2e8d0138c1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:30.976366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b1dd9-db4c-495d-9e99-2e8d0138c1fd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:30.976841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b1dd9-db4c-495d-9e99-2e8d0138c1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:30.977323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b1dd9-db4c-495d-9e99-2e8d0138c1fd None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:30.978228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b1dd9-db4c-495d-9e99-2e8d0138c1fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:30.978742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b1dd9-db4c-495d-9e99-2e8d0138c1fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:30.980346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b1dd9-db4c-495d-9e99-2e8d0138c1fd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:30.992889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 369/754] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:30.993989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b1dd9-db4c-495d-9e99-2e8d0138c1fd None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:31.030763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:31.083367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748da66-63c9-4d41-95fe-5ae5f7ff8855 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:31.102523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b1dd9-db4c-495d-9e99-2e8d0138c1fd None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:31.126276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:31.131761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a30b0e-a815-43d1-b917-d4b38e194931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:31.132845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30b0e-a815-43d1-b917-d4b38e194931 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.133259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30b0e-a815-43d1-b917-d4b38e194931 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.133945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30b0e-a815-43d1-b917-d4b38e194931 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.142433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593145b3-eb3b-4441-b403-b5f450a0fbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:31.143139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593145b3-eb3b-4441-b403-b5f450a0fbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.143305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a30b0e-a815-43d1-b917-d4b38e194931 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:31.143597 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593145b3-eb3b-4441-b403-b5f450a0fbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.143976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593145b3-eb3b-4441-b403-b5f450a0fbc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.145182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a30b0e-a815-43d1-b917-d4b38e194931 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-83a30b0e-a815-43d1-b917-d4b38e194931 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:31.161312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:31.162403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:31.163526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:31.163526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:31.163526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:31.163526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:31.163526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:31.163526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:31.163526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:31.163526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:31.165955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 386/755] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:31.166992 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a30b0e-a815-43d1-b917-d4b38e194931 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:31.222146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593145b3-eb3b-4441-b403-b5f450a0fbc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:31.223693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593145b3-eb3b-4441-b403-b5f450a0fbc0 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:31.300351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a30b0e-a815-43d1-b917-d4b38e194931 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:31.330978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 370/756] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:09:31.330978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593145b3-eb3b-4441-b403-b5f450a0fbc0 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:31.331825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:31.350859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d489ba0-d32b-4196-8840-1c50e1a5f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:31.351727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d489ba0-d32b-4196-8840-1c50e1a5f88f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.352143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d489ba0-d32b-4196-8840-1c50e1a5f88f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.352550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d489ba0-d32b-4196-8840-1c50e1a5f88f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.362177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d489ba0-d32b-4196-8840-1c50e1a5f88f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:31.364069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d489ba0-d32b-4196-8840-1c50e1a5f88f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:31.366304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-4d489ba0-d32b-4196-8840-1c50e1a5f88f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:31.378895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:31.379934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:31.381000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:31.381000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:31.381000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:31.381000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:31.381000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:31.381000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:31.381000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:31.381000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:31.389321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 387/757] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:31.389321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d489ba0-d32b-4196-8840-1c50e1a5f88f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:31.442543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593145b3-eb3b-4441-b403-b5f450a0fbc0 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:31.477208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4710206-e1c0-4bc3-aeeb-054e3a0e9b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:31.477760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4710206-e1c0-4bc3-aeeb-054e3a0e9b2c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.478034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4710206-e1c0-4bc3-aeeb-054e3a0e9b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.478300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4710206-e1c0-4bc3-aeeb-054e3a0e9b2c None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.479064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4710206-e1c0-4bc3-aeeb-054e3a0e9b2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:31.479331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4710206-e1c0-4bc3-aeeb-054e3a0e9b2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:31.480850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4710206-e1c0-4bc3-aeeb-054e3a0e9b2c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:31.492704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 371/758] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:31.493551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4710206-e1c0-4bc3-aeeb-054e3a0e9b2c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:31.525900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d489ba0-d32b-4196-8840-1c50e1a5f88f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:31.566082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada303b-338b-48a2-9797-7a53f2cbefbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:31.567105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada303b-338b-48a2-9797-7a53f2cbefbd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.567459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada303b-338b-48a2-9797-7a53f2cbefbd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.567803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada303b-338b-48a2-9797-7a53f2cbefbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.582232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada303b-338b-48a2-9797-7a53f2cbefbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:31.583763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada303b-338b-48a2-9797-7a53f2cbefbd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:31.593881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-eada303b-338b-48a2-9797-7a53f2cbefbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:31.605226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:31.606213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:31.607056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:31.607056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:31.607056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:31.609655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 388/759] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:31.611754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada303b-338b-48a2-9797-7a53f2cbefbd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:31.623702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4710206-e1c0-4bc3-aeeb-054e3a0e9b2c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:31.651908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:31.672136 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27654f1-39c6-4e6e-9cfa-e11db37a7607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:31.673116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27654f1-39c6-4e6e-9cfa-e11db37a7607 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.673639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27654f1-39c6-4e6e-9cfa-e11db37a7607 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.674109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27654f1-39c6-4e6e-9cfa-e11db37a7607 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.674990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27654f1-39c6-4e6e-9cfa-e11db37a7607 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:31.675488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27654f1-39c6-4e6e-9cfa-e11db37a7607 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:31.677110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27654f1-39c6-4e6e-9cfa-e11db37a7607 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:31.690438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 372/760] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:31.691633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27654f1-39c6-4e6e-9cfa-e11db37a7607 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:31.741705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada303b-338b-48a2-9797-7a53f2cbefbd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:31.777492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccd2d9-e540-4e41-8b19-ab5dc25db302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:31.777972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccd2d9-e540-4e41-8b19-ab5dc25db302 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.778249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccd2d9-e540-4e41-8b19-ab5dc25db302 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.778519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccd2d9-e540-4e41-8b19-ab5dc25db302 None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/286f4e203e7a4a55bca0a58d061af2bd/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.780417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ccd2d9-e540-4e41-8b19-ab5dc25db302 None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=286f4e203e7a4a55bca0a58d061af2bd, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:31.781865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ccd2d9-e540-4e41-8b19-ab5dc25db302 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:31.798363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 389/761] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:09:31 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/286f4e203e7a4a55bca0a58d061af2bd/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:09:31.799305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccd2d9-e540-4e41-8b19-ab5dc25db302 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:31.806029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27654f1-39c6-4e6e-9cfa-e11db37a7607 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:31.845772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90f724-b0a3-4d2e-8b03-268c427d1fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:31.846764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f724-b0a3-4d2e-8b03-268c427d1fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.847270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f724-b0a3-4d2e-8b03-268c427d1fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.847756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f724-b0a3-4d2e-8b03-268c427d1fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.848682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90f724-b0a3-4d2e-8b03-268c427d1fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:31.849193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90f724-b0a3-4d2e-8b03-268c427d1fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:31.850836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90f724-b0a3-4d2e-8b03-268c427d1fa3 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:31.861278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:31.865848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 373/762] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:31.871240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0468d8-1cb7-47a8-b536-34c759b9f82a None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.873105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0468d8-1cb7-47a8-b536-34c759b9f82a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.873696 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0468d8-1cb7-47a8-b536-34c759b9f82a None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.879567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 374/763] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:31 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:31.880699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0468d8-1cb7-47a8-b536-34c759b9f82a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:31.931796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:31.946061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccd2d9-e540-4e41-8b19-ab5dc25db302 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:31.993561 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b3b72-d252-4369-b745-b138d18ec250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:31.994202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b3b72-d252-4369-b745-b138d18ec250 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:31.994757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b3b72-d252-4369-b745-b138d18ec250 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:31.994877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b3b72-d252-4369-b745-b138d18ec250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:31.995546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b3b72-d252-4369-b745-b138d18ec250 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=f8dec1dbdcd94316aad25c93c313a5a5) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:31.995857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b3b72-d252-4369-b745-b138d18ec250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:31.997493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b3b72-d252-4369-b745-b138d18ec250 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:32.005240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0468d8-1cb7-47a8-b536-34c759b9f82a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:32.007081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 390/764] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:31 2022] GET /identity/v3/role_assignments?user.id=f8dec1dbdcd94316aad25c93c313a5a5&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=36331ec562ff42f883fc883363bc4c8d => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:32.008299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab9429-25ad-456a-8581-aac351035668 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:32.008585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab9429-25ad-456a-8581-aac351035668 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:32.008855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab9429-25ad-456a-8581-aac351035668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:32.040279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7aab9429-25ad-456a-8581-aac351035668 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:32.041533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7aab9429-25ad-456a-8581-aac351035668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:32.060302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e9b59-e51f-4611-97a6-4fb98440d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:32.060982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e9b59-e51f-4611-97a6-4fb98440d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:32.061306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e9b59-e51f-4611-97a6-4fb98440d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:32.061667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e9b59-e51f-4611-97a6-4fb98440d4d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:32.070741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e9b59-e51f-4611-97a6-4fb98440d4d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:32.072251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e9b59-e51f-4611-97a6-4fb98440d4d3 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-ae5e9b59-e51f-4611-97a6-4fb98440d4d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:32.085185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:32.086369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:32.087439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:32.087439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:32.087439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:32.087439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:32.087439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:32.087439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:32.087439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:32.087439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:32.091204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 375/765] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:32.092650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e9b59-e51f-4611-97a6-4fb98440d4d3 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:32.134089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 391/766] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:32.177965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:32.207846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e9b59-e51f-4611-97a6-4fb98440d4d3 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:32.246440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887933e-2186-4c63-a8e3-3cc2c91cc888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:32.247293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887933e-2186-4c63-a8e3-3cc2c91cc888 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:32.247732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887933e-2186-4c63-a8e3-3cc2c91cc888 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:32.248146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887933e-2186-4c63-a8e3-3cc2c91cc888 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:32.248974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887933e-2186-4c63-a8e3-3cc2c91cc888 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:32.249427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887933e-2186-4c63-a8e3-3cc2c91cc888 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:32.251660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887933e-2186-4c63-a8e3-3cc2c91cc888 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:32.265804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 376/767] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:32.272751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab9429-25ad-456a-8581-aac351035668 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:32.320250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435c2d4-dec3-4b90-a61f-8faf22657106 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:32.320597 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435c2d4-dec3-4b90-a61f-8faf22657106 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:32.320800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435c2d4-dec3-4b90-a61f-8faf22657106 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:32.322419 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 377/768] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:09:32.327973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d061e-4605-4c0f-9b51-fe8ef3f3dcfc None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:32.328498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d061e-4605-4c0f-9b51-fe8ef3f3dcfc None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:32.329144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d061e-4605-4c0f-9b51-fe8ef3f3dcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:32.350471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-522d061e-4605-4c0f-9b51-fe8ef3f3dcfc None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:32.352564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-522d061e-4605-4c0f-9b51-fe8ef3f3dcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:32.409878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:32.450927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab9429-25ad-456a-8581-aac351035668 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:32.484725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 378/769] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:09:32.490445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d061e-4605-4c0f-9b51-fe8ef3f3dcfc None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:32.516486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4080b22-86ee-47b4-b561-6b0966d1cefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:32.517026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4080b22-86ee-47b4-b561-6b0966d1cefd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:32.517300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4080b22-86ee-47b4-b561-6b0966d1cefd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:32.517601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4080b22-86ee-47b4-b561-6b0966d1cefd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:32.518327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4080b22-86ee-47b4-b561-6b0966d1cefd None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=2314d9416eea4856aec5ac2a0cdfdfc1) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:32.518631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4080b22-86ee-47b4-b561-6b0966d1cefd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:32.520054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4080b22-86ee-47b4-b561-6b0966d1cefd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:32.530677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 392/770] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:32 2022] GET /identity/v3/role_assignments?user.id=2314d9416eea4856aec5ac2a0cdfdfc1&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:32.574939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:32.644064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d061e-4605-4c0f-9b51-fe8ef3f3dcfc None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:32.695602 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53404401-a156-40ba-b338-c7a414d1f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:32.696247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53404401-a156-40ba-b338-c7a414d1f102 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:32.696524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53404401-a156-40ba-b338-c7a414d1f102 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:32.696930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53404401-a156-40ba-b338-c7a414d1f102 None admin] PATH_INFO: `/v3/services/network` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:32.697580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53404401-a156-40ba-b338-c7a414d1f102 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:32.698991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53404401-a156-40ba-b338-c7a414d1f102 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-53404401-a156-40ba-b338-c7a414d1f102 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:32.709315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:32.710525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:32.711809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 379/771] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:32 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:32.718916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4080b22-86ee-47b4-b561-6b0966d1cefd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:32.824148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53404401-a156-40ba-b338-c7a414d1f102 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:32.900290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4080b22-86ee-47b4-b561-6b0966d1cefd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:32.964804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c08995-1a80-4032-8b6f-6efc820e3da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:32.965474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c08995-1a80-4032-8b6f-6efc820e3da6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:32.965780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c08995-1a80-4032-8b6f-6efc820e3da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:32.966050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c08995-1a80-4032-8b6f-6efc820e3da6 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:32.966749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c08995-1a80-4032-8b6f-6efc820e3da6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:32.967048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c08995-1a80-4032-8b6f-6efc820e3da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:32.968465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c08995-1a80-4032-8b6f-6efc820e3da6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:32.978990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 393/772] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:09:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:32.984515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c08995-1a80-4032-8b6f-6efc820e3da6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:32.987918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53404401-a156-40ba-b338-c7a414d1f102 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:33.009580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:33.011878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:33.052384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddc738-29c1-42c5-9c0b-3dd97ba4f7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:33.053020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddc738-29c1-42c5-9c0b-3dd97ba4f7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:33.053296 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddc738-29c1-42c5-9c0b-3dd97ba4f7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:33.053583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddc738-29c1-42c5-9c0b-3dd97ba4f7c9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:33.054170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddc738-29c1-42c5-9c0b-3dd97ba4f7c9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:33.055539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddc738-29c1-42c5-9c0b-3dd97ba4f7c9 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-15ddc738-29c1-42c5-9c0b-3dd97ba4f7c9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:33.070400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:33.071393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 20:09:33.072239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:33.072959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 380/773] 10.222.0.23 () {56 vars in 1129 bytes} [Wed Oct 5 20:09:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:33.080849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddc738-29c1-42c5-9c0b-3dd97ba4f7c9 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:33.173436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c08995-1a80-4032-8b6f-6efc820e3da6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:33.261560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddc738-29c1-42c5-9c0b-3dd97ba4f7c9 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:33.264948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7aa47c-4679-44fe-9a3f-fbd7d128186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:33.265581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7aa47c-4679-44fe-9a3f-fbd7d128186e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:33.266000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7aa47c-4679-44fe-9a3f-fbd7d128186e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:33.266303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7aa47c-4679-44fe-9a3f-fbd7d128186e None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:33.266934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7aa47c-4679-44fe-9a3f-fbd7d128186e None admin] RBAC: Adding query filter params (type=network) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:33.267236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7aa47c-4679-44fe-9a3f-fbd7d128186e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:33.269450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7aa47c-4679-44fe-9a3f-fbd7d128186e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:33.280659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 394/774] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:09:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:33.288279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7aa47c-4679-44fe-9a3f-fbd7d128186e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:33.346321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04eff0-9fde-4b88-b33b-aa8528b30e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:33.346949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04eff0-9fde-4b88-b33b-aa8528b30e54 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:33.347230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04eff0-9fde-4b88-b33b-aa8528b30e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:33.347501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04eff0-9fde-4b88-b33b-aa8528b30e54 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:33.348250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04eff0-9fde-4b88-b33b-aa8528b30e54 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:33.348558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04eff0-9fde-4b88-b33b-aa8528b30e54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:33.350343 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04eff0-9fde-4b88-b33b-aa8528b30e54 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:33.365262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 381/775] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 20:09:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:33.371005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04eff0-9fde-4b88-b33b-aa8528b30e54 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:33.378562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:33.404171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7aa47c-4679-44fe-9a3f-fbd7d128186e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:33.472182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b6c3d-b6fa-4ae9-b37f-8c5bb257b011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:33.474010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:33.474691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b6c3d-b6fa-4ae9-b37f-8c5bb257b011 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:33.474972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b6c3d-b6fa-4ae9-b37f-8c5bb257b011 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:33.475238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b6c3d-b6fa-4ae9-b37f-8c5bb257b011 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:33.475956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b6c3d-b6fa-4ae9-b37f-8c5bb257b011 None admin] RBAC: Adding query filter params (interface=public, service_id=82bc1f204ea54af2ae42317716eb1156, region_id=RegionOne) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:33.476256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b6c3d-b6fa-4ae9-b37f-8c5bb257b011 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:33.477655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b6c3d-b6fa-4ae9-b37f-8c5bb257b011 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:33.497284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 395/776] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 20:09:33 2022] GET /identity/v3/endpoints?service_id=82bc1f204ea54af2ae42317716eb1156&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:33.501733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b6c3d-b6fa-4ae9-b37f-8c5bb257b011 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:33.557545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04eff0-9fde-4b88-b33b-aa8528b30e54 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:33.604374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68952bd-7964-47c7-a914-54972879f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:33.604908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68952bd-7964-47c7-a914-54972879f4bd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:33.605187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68952bd-7964-47c7-a914-54972879f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:33.605458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68952bd-7964-47c7-a914-54972879f4bd None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:33.606116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68952bd-7964-47c7-a914-54972879f4bd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:33.606415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68952bd-7964-47c7-a914-54972879f4bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:33.608654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68952bd-7964-47c7-a914-54972879f4bd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:33.617192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 382/777] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 20:09:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:33.624713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68952bd-7964-47c7-a914-54972879f4bd None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:33.717070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b6c3d-b6fa-4ae9-b37f-8c5bb257b011 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:33.748889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68952bd-7964-47c7-a914-54972879f4bd None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:33.757807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:33.791588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909518f1-827f-4d18-a744-0181c308084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:33.792201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909518f1-827f-4d18-a744-0181c308084d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:33.792479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909518f1-827f-4d18-a744-0181c308084d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:33.792782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909518f1-827f-4d18-a744-0181c308084d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:33.793445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909518f1-827f-4d18-a744-0181c308084d None admin] RBAC: Adding query filter params (interface=public, service_id=d6d3c49685204d0c85ba855bbedcf7bc, region_id=RegionOne) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:33.793798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909518f1-827f-4d18-a744-0181c308084d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:33.798965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909518f1-827f-4d18-a744-0181c308084d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:33.813272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 383/778] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 20:09:33 2022] GET /identity/v3/endpoints?service_id=d6d3c49685204d0c85ba855bbedcf7bc&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:33.819272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909518f1-827f-4d18-a744-0181c308084d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:33.822576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c121c55-5296-466f-a195-b74d0a66f942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:33.823944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c121c55-5296-466f-a195-b74d0a66f942 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:33.825677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c121c55-5296-466f-a195-b74d0a66f942 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:33.827134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c121c55-5296-466f-a195-b74d0a66f942 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:33.829274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c121c55-5296-466f-a195-b74d0a66f942 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:33.831081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c121c55-5296-466f-a195-b74d0a66f942 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:33.843793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 396/779] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 20:09:33 2022] GET /identity/v3/services => generated 772 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:33.980131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909518f1-827f-4d18-a744-0181c308084d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:34.027205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f45d40-d785-4138-9811-6f1985ac3163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:34.028192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f45d40-d785-4138-9811-6f1985ac3163 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:34.028695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f45d40-d785-4138-9811-6f1985ac3163 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:34.029190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f45d40-d785-4138-9811-6f1985ac3163 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:34.030049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f45d40-d785-4138-9811-6f1985ac3163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:34.031677 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f45d40-d785-4138-9811-6f1985ac3163 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:34.037818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 384/780] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 20:09:33 2022] GET /identity/v3/services => generated 772 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:34.486016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617e38a-7965-4e2b-a06f-5440062f9931 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:34.487032 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617e38a-7965-4e2b-a06f-5440062f9931 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:34.487541 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617e38a-7965-4e2b-a06f-5440062f9931 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:34.489415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 397/781] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:34.494888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616aa45a-7d56-4380-b28a-cb6217e7a256 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:34.495401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616aa45a-7d56-4380-b28a-cb6217e7a256 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:34.495897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616aa45a-7d56-4380-b28a-cb6217e7a256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:34.522510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-616aa45a-7d56-4380-b28a-cb6217e7a256 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:34.524059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-616aa45a-7d56-4380-b28a-cb6217e7a256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:34.562838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:34.614956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 385/782] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:34.779435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2365d-f801-4e23-93be-5cca474938ca None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:34.780041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2365d-f801-4e23-93be-5cca474938ca None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:34.780516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2365d-f801-4e23-93be-5cca474938ca None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:34.782731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 398/783] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:34.788111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0c323-2ebd-46d3-8ffc-67b2dc84610f None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:34.788692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0c323-2ebd-46d3-8ffc-67b2dc84610f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:34.789186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0c323-2ebd-46d3-8ffc-67b2dc84610f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:34.812294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-4fc0c323-2ebd-46d3-8ffc-67b2dc84610f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:34.813945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4fc0c323-2ebd-46d3-8ffc-67b2dc84610f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:34.909805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 386/784] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:34.930600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2365d-f801-4e23-93be-5cca474938ca None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:34.937108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:35.053568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2365d-f801-4e23-93be-5cca474938ca None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:35.089180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cba48-7dd8-47e3-abcc-680a63178caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:35.089861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cba48-7dd8-47e3-abcc-680a63178caf None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:35.090143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cba48-7dd8-47e3-abcc-680a63178caf None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:35.090410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cba48-7dd8-47e3-abcc-680a63178caf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:35.102436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cba48-7dd8-47e3-abcc-680a63178caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-3f3cba48-7dd8-47e3-abcc-680a63178caf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:35.107891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:35.108882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:35.109752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 399/785] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:35.115786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0c323-2ebd-46d3-8ffc-67b2dc84610f None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:35.209092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cba48-7dd8-47e3-abcc-680a63178caf None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:35.298477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:35.299183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:35.303905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0c323-2ebd-46d3-8ffc-67b2dc84610f None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:35.343767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cba48-7dd8-47e3-abcc-680a63178caf None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:35.350652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ae939-66cf-4dba-a7a2-f8645239adc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:35.351482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ae939-66cf-4dba-a7a2-f8645239adc5 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:35.351939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ae939-66cf-4dba-a7a2-f8645239adc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:35.352392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ae939-66cf-4dba-a7a2-f8645239adc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:35.353256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ae939-66cf-4dba-a7a2-f8645239adc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:35.353765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ae939-66cf-4dba-a7a2-f8645239adc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:35.355327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ae939-66cf-4dba-a7a2-f8645239adc5 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:35.364769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 387/786] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:35.366251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ba7b9-00cc-4183-b0ac-7967a8564d28 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:35.366736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ba7b9-00cc-4183-b0ac-7967a8564d28 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:35.367195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ba7b9-00cc-4183-b0ac-7967a8564d28 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:35.372729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 388/787] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:35.373760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ba7b9-00cc-4183-b0ac-7967a8564d28 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:35.390234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4ada1-9998-4b5b-bdff-f077f434f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:35.391103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4ada1-9998-4b5b-bdff-f077f434f22c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:35.391255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4ada1-9998-4b5b-bdff-f077f434f22c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:35.391520 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4ada1-9998-4b5b-bdff-f077f434f22c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:35.403985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4ada1-9998-4b5b-bdff-f077f434f22c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-71d4ada1-9998-4b5b-bdff-f077f434f22c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:35.410724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:35.411672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:35.412505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:35.416962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 400/788] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:35.419119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdd7d-6808-415c-8187-03ed0936a6bd None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:35.419388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdd7d-6808-415c-8187-03ed0936a6bd None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:35.419650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaefdd7d-6808-415c-8187-03ed0936a6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:35.443824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-eaefdd7d-6808-415c-8187-03ed0936a6bd None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:35.444888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-eaefdd7d-6808-415c-8187-03ed0936a6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:35.498732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ba7b9-00cc-4183-b0ac-7967a8564d28 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:35.533199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 401/789] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:35.533199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdd7d-6808-415c-8187-03ed0936a6bd None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:35.559308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ea8b2-7d52-4d2b-895f-46ca926b1e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:35.559866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ea8b2-7d52-4d2b-895f-46ca926b1e59 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:35.560145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ea8b2-7d52-4d2b-895f-46ca926b1e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:35.560667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ea8b2-7d52-4d2b-895f-46ca926b1e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:35.583172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ea8b2-7d52-4d2b-895f-46ca926b1e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:35.584625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ea8b2-7d52-4d2b-895f-46ca926b1e59 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-b29ea8b2-7d52-4d2b-895f-46ca926b1e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:35.612660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:35.613676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:35.614553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:35.614553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:35.614553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:35.614553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 389/790] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:35.619731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ea8b2-7d52-4d2b-895f-46ca926b1e59 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:35.638184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:35.639196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:35.723392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaefdd7d-6808-415c-8187-03ed0936a6bd None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:35.803268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2705a-a21f-48c4-853b-449acdfe33cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:35.804366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2705a-a21f-48c4-853b-449acdfe33cd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:35.804876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2705a-a21f-48c4-853b-449acdfe33cd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:35.805347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2705a-a21f-48c4-853b-449acdfe33cd None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:35.806233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2705a-a21f-48c4-853b-449acdfe33cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:35.806426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ea8b2-7d52-4d2b-895f-46ca926b1e59 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:35.806933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2705a-a21f-48c4-853b-449acdfe33cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:35.808783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2705a-a21f-48c4-853b-449acdfe33cd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:35.827021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 402/791] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:35.832161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2705a-a21f-48c4-853b-449acdfe33cd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:35.841359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d172b-8e0b-4056-aaa3-bd5780f7422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:35.842001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d172b-8e0b-4056-aaa3-bd5780f7422f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:35.846582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d172b-8e0b-4056-aaa3-bd5780f7422f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:35.846582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d172b-8e0b-4056-aaa3-bd5780f7422f None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:35.846582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d172b-8e0b-4056-aaa3-bd5780f7422f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:35.846582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d172b-8e0b-4056-aaa3-bd5780f7422f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:35.846582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d172b-8e0b-4056-aaa3-bd5780f7422f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:35.858799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 390/792] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:35.865564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d172b-8e0b-4056-aaa3-bd5780f7422f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:35.935288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:35.961562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2705a-a21f-48c4-853b-449acdfe33cd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:35.972797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d172b-8e0b-4056-aaa3-bd5780f7422f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:35.980247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:36.001855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128f6c0-6f9c-432c-ab84-adcad0628db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.002889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128f6c0-6f9c-432c-ab84-adcad0628db1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.003373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128f6c0-6f9c-432c-ab84-adcad0628db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.003872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128f6c0-6f9c-432c-ab84-adcad0628db1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.010000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1842f92-1bdf-47b7-a5d8-f5706f20bd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.010630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1842f92-1bdf-47b7-a5d8-f5706f20bd86 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.010910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1842f92-1bdf-47b7-a5d8-f5706f20bd86 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.011184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1842f92-1bdf-47b7-a5d8-f5706f20bd86 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.017168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128f6c0-6f9c-432c-ab84-adcad0628db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.018916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128f6c0-6f9c-432c-ab84-adcad0628db1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.019490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1842f92-1bdf-47b7-a5d8-f5706f20bd86 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.021036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1842f92-1bdf-47b7-a5d8-f5706f20bd86 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-a1842f92-1bdf-47b7-a5d8-f5706f20bd86 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:36.033997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:36.034943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:09:36.035875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-9128f6c0-6f9c-432c-ab84-adcad0628db1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:36.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:36.038796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:36.039658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:36.039658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:36.039658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:36.040194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 391/793] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 20:09:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:36.040742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1842f92-1bdf-47b7-a5d8-f5706f20bd86 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:36.042019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 403/794] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:36.043068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128f6c0-6f9c-432c-ab84-adcad0628db1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:36.147468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1842f92-1bdf-47b7-a5d8-f5706f20bd86 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:36.153248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128f6c0-6f9c-432c-ab84-adcad0628db1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:36.176831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:36.187878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84feebc4-a56f-4ac2-9905-1b33fb778a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.188978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84feebc4-a56f-4ac2-9905-1b33fb778a08 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.190144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84feebc4-a56f-4ac2-9905-1b33fb778a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.190905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84feebc4-a56f-4ac2-9905-1b33fb778a08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.194574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b10b6-feed-40fd-9414-213d78dba655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.195177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b10b6-feed-40fd-9414-213d78dba655 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.195450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b10b6-feed-40fd-9414-213d78dba655 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.195711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b10b6-feed-40fd-9414-213d78dba655 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.196424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b10b6-feed-40fd-9414-213d78dba655 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:36.196723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b10b6-feed-40fd-9414-213d78dba655 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.198153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b10b6-feed-40fd-9414-213d78dba655 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.204989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84feebc4-a56f-4ac2-9905-1b33fb778a08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-84feebc4-a56f-4ac2-9905-1b33fb778a08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:36.211686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:36.212614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:36.213429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 392/795] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:36.220661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84feebc4-a56f-4ac2-9905-1b33fb778a08 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:36.225701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 404/796] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:36.229544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b10b6-feed-40fd-9414-213d78dba655 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:36.264454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:36.353276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84feebc4-a56f-4ac2-9905-1b33fb778a08 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:36.354896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b10b6-feed-40fd-9414-213d78dba655 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:36.390989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035c5ac-db27-4643-97ee-94ea02de7b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.391759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035c5ac-db27-4643-97ee-94ea02de7b33 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.391871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035c5ac-db27-4643-97ee-94ea02de7b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.392146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035c5ac-db27-4643-97ee-94ea02de7b33 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.392779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035c5ac-db27-4643-97ee-94ea02de7b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:36.393076 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035c5ac-db27-4643-97ee-94ea02de7b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.394464 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035c5ac-db27-4643-97ee-94ea02de7b33 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.405818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01656886-795f-4bf2-bcfa-20887915866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.407514 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01656886-795f-4bf2-bcfa-20887915866e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.408011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01656886-795f-4bf2-bcfa-20887915866e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.408484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01656886-795f-4bf2-bcfa-20887915866e None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.409347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01656886-795f-4bf2-bcfa-20887915866e None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:36.409868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01656886-795f-4bf2-bcfa-20887915866e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.411519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01656886-795f-4bf2-bcfa-20887915866e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.414470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 393/797] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:36.415354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035c5ac-db27-4643-97ee-94ea02de7b33 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:36.427887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 405/798] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:36.427887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01656886-795f-4bf2-bcfa-20887915866e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:36.477449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:36.524762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:36.532656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035c5ac-db27-4643-97ee-94ea02de7b33 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:36.540832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01656886-795f-4bf2-bcfa-20887915866e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:36.569246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab9117-ad9c-4e67-b40d-3b6e169c450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.569908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab9117-ad9c-4e67-b40d-3b6e169c450f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.570189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab9117-ad9c-4e67-b40d-3b6e169c450f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.570461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab9117-ad9c-4e67-b40d-3b6e169c450f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.579730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b77d0-a3c5-4cd4-b0d2-3f190e09fd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.580349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b77d0-a3c5-4cd4-b0d2-3f190e09fd5b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.580623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b77d0-a3c5-4cd4-b0d2-3f190e09fd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.580898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b77d0-a3c5-4cd4-b0d2-3f190e09fd5b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.589068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b77d0-a3c5-4cd4-b0d2-3f190e09fd5b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.590611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47b77d0-a3c5-4cd4-b0d2-3f190e09fd5b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-f47b77d0-a3c5-4cd4-b0d2-3f190e09fd5b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:36.602139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:36.603061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:36.604046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:36.608167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 406/799] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:36.609009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b77d0-a3c5-4cd4-b0d2-3f190e09fd5b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:36.639175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab9117-ad9c-4e67-b40d-3b6e169c450f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.640634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab9117-ad9c-4e67-b40d-3b6e169c450f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.688189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:36.711033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b77d0-a3c5-4cd4-b0d2-3f190e09fd5b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:36.740418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 394/800] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:09:36.741729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eae208-f3a6-4aba-b21a-00833495a6b0 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.742014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eae208-f3a6-4aba-b21a-00833495a6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.742287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eae208-f3a6-4aba-b21a-00833495a6b0 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.745642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 395/801] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:36.745642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eae208-f3a6-4aba-b21a-00833495a6b0 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:36.746250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd39e2-386a-4d6c-90e5-41e35472f55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.746875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd39e2-386a-4d6c-90e5-41e35472f55c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.747150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd39e2-386a-4d6c-90e5-41e35472f55c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.747416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd39e2-386a-4d6c-90e5-41e35472f55c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.759285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd39e2-386a-4d6c-90e5-41e35472f55c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.760783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd39e2-386a-4d6c-90e5-41e35472f55c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.769735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-25cd39e2-386a-4d6c-90e5-41e35472f55c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:36.777120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:36.778078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:36.778933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:36.778933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:36.778933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:36.778933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 407/802] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:36.779602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3df7b-6a49-4b4e-9ff2-33f3ef025a30 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.779818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3df7b-6a49-4b4e-9ff2-33f3ef025a30 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.780088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3df7b-6a49-4b4e-9ff2-33f3ef025a30 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.781306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 408/803] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:36.782061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3df7b-6a49-4b4e-9ff2-33f3ef025a30 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:36.854653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eae208-f3a6-4aba-b21a-00833495a6b0 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:36.885284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3df7b-6a49-4b4e-9ff2-33f3ef025a30 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:36.889268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc77d54-2ad4-4634-a0cd-bd7008415c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.889902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc77d54-2ad4-4634-a0cd-bd7008415c0f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.890183 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc77d54-2ad4-4634-a0cd-bd7008415c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.890456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc77d54-2ad4-4634-a0cd-bd7008415c0f None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.891209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc77d54-2ad4-4634-a0cd-bd7008415c0f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:36.891513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc77d54-2ad4-4634-a0cd-bd7008415c0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.892942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc77d54-2ad4-4634-a0cd-bd7008415c0f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.909932 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 396/804] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:36.911234 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01336e00-2119-496b-a021-51bd02db838c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.911519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01336e00-2119-496b-a021-51bd02db838c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.911790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01336e00-2119-496b-a021-51bd02db838c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.918819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bb607-b39f-4c53-82c7-22aadfdbd80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:36.919444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bb607-b39f-4c53-82c7-22aadfdbd80e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.919718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bb607-b39f-4c53-82c7-22aadfdbd80e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.919986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bb607-b39f-4c53-82c7-22aadfdbd80e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.924302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:36.929740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-01336e00-2119-496b-a021-51bd02db838c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:36.930938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-01336e00-2119-496b-a021-51bd02db838c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:36.931348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bb607-b39f-4c53-82c7-22aadfdbd80e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:36.932823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bb607-b39f-4c53-82c7-22aadfdbd80e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-e91bb607-b39f-4c53-82c7-22aadfdbd80e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:36.947733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:36.948616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:36.949492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:36.949492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:36.949492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:36.949492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 409/805] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:36.950080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac10cc-6fbc-4e15-888f-4db772b0bd4c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:36.950173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac10cc-6fbc-4e15-888f-4db772b0bd4c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:36.950419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac10cc-6fbc-4e15-888f-4db772b0bd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:36.967815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9dac10cc-6fbc-4e15-888f-4db772b0bd4c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:36.969057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9dac10cc-6fbc-4e15-888f-4db772b0bd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:36.998208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 397/806] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:36.998867 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01336e00-2119-496b-a021-51bd02db838c None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.023974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:37.049891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 410/807] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:37.049891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac10cc-6fbc-4e15-888f-4db772b0bd4c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.115232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01336e00-2119-496b-a021-51bd02db838c None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.150434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e940a-59aa-4807-82e5-68d9d23c745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.151186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e940a-59aa-4807-82e5-68d9d23c745e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.151534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e940a-59aa-4807-82e5-68d9d23c745e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.151879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e940a-59aa-4807-82e5-68d9d23c745e None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.152592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e940a-59aa-4807-82e5-68d9d23c745e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:37.152969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e940a-59aa-4807-82e5-68d9d23c745e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.154427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e940a-59aa-4807-82e5-68d9d23c745e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.157660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:37.178171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 398/808] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:37.178171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e940a-59aa-4807-82e5-68d9d23c745e None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.198678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac10cc-6fbc-4e15-888f-4db772b0bd4c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.243370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d15a99-73e2-4343-bb37-8f36515358e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.244219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d15a99-73e2-4343-bb37-8f36515358e7 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.244677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d15a99-73e2-4343-bb37-8f36515358e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.245089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d15a99-73e2-4343-bb37-8f36515358e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.258282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d15a99-73e2-4343-bb37-8f36515358e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.260478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d15a99-73e2-4343-bb37-8f36515358e7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-03d15a99-73e2-4343-bb37-8f36515358e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:37.279443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:37.280385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:37.281666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:37.281666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:37.281666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:37.282383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e940a-59aa-4807-82e5-68d9d23c745e None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.282455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 411/809] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:37.282455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d15a99-73e2-4343-bb37-8f36515358e7 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.318472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:37.326030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57fda2-9cb6-495e-b824-e87aba1d9594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.326644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57fda2-9cb6-495e-b824-e87aba1d9594 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.326947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57fda2-9cb6-495e-b824-e87aba1d9594 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.327219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57fda2-9cb6-495e-b824-e87aba1d9594 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.327860 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57fda2-9cb6-495e-b824-e87aba1d9594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:37.328170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57fda2-9cb6-495e-b824-e87aba1d9594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.329588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57fda2-9cb6-495e-b824-e87aba1d9594 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.340997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 399/810] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:37.341827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57fda2-9cb6-495e-b824-e87aba1d9594 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.385677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d15a99-73e2-4343-bb37-8f36515358e7 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.400669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:37.418558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fd02f-956f-4229-938f-996da45853af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.419173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fd02f-956f-4229-938f-996da45853af None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.419445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fd02f-956f-4229-938f-996da45853af None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.419711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fd02f-956f-4229-938f-996da45853af None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.426927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fd02f-956f-4229-938f-996da45853af None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.428337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fd02f-956f-4229-938f-996da45853af None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-c41fd02f-956f-4229-938f-996da45853af None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:37.439845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:37.440808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 412/811] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:37.442101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fd02f-956f-4229-938f-996da45853af None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.443733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57fda2-9cb6-495e-b824-e87aba1d9594 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.477427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11cec0-0846-4412-b146-8aa192d169c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.478015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cec0-0846-4412-b146-8aa192d169c7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.478290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cec0-0846-4412-b146-8aa192d169c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.478564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11cec0-0846-4412-b146-8aa192d169c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.479137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11cec0-0846-4412-b146-8aa192d169c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.480495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11cec0-0846-4412-b146-8aa192d169c7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-7d11cec0-0846-4412-b146-8aa192d169c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:37.488999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:37.490012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:37.490820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 400/812] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/services/network => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:37.491571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11cec0-0846-4412-b146-8aa192d169c7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.534401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:37.541559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fd02f-956f-4229-938f-996da45853af None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.575636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98e11b-4f4b-4a7d-9b04-b3c091edbb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.576228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98e11b-4f4b-4a7d-9b04-b3c091edbb9a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.576497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98e11b-4f4b-4a7d-9b04-b3c091edbb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.576768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98e11b-4f4b-4a7d-9b04-b3c091edbb9a None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.577392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98e11b-4f4b-4a7d-9b04-b3c091edbb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:37.577715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98e11b-4f4b-4a7d-9b04-b3c091edbb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.579099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98e11b-4f4b-4a7d-9b04-b3c091edbb9a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.590899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11cec0-0846-4412-b146-8aa192d169c7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.591026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 413/813] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:37.591708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98e11b-4f4b-4a7d-9b04-b3c091edbb9a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.610994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:37.625317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49810b7-a680-4a85-9611-588e0ed0b68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.626536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49810b7-a680-4a85-9611-588e0ed0b68d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.626814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49810b7-a680-4a85-9611-588e0ed0b68d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.627077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49810b7-a680-4a85-9611-588e0ed0b68d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.627644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49810b7-a680-4a85-9611-588e0ed0b68d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.629004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49810b7-a680-4a85-9611-588e0ed0b68d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-f49810b7-a680-4a85-9611-588e0ed0b68d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:37.637404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:37.638351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:37.639429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 401/814] 10.222.0.23 () {56 vars in 1129 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:37.640358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49810b7-a680-4a85-9611-588e0ed0b68d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.700357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98e11b-4f4b-4a7d-9b04-b3c091edbb9a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.733070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763c210-7699-476b-9a88-807ca6eb68dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.733650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763c210-7699-476b-9a88-807ca6eb68dc None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.733930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763c210-7699-476b-9a88-807ca6eb68dc None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.734200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763c210-7699-476b-9a88-807ca6eb68dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.734622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49810b7-a680-4a85-9611-588e0ed0b68d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.741362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763c210-7699-476b-9a88-807ca6eb68dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.742813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763c210-7699-476b-9a88-807ca6eb68dc None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-b763c210-7699-476b-9a88-807ca6eb68dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:37.754332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:37.755289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 414/815] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:37.756150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763c210-7699-476b-9a88-807ca6eb68dc None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.772562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b414c-d7d8-4f81-9292-79529c7cc544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.772900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:37.773161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b414c-d7d8-4f81-9292-79529c7cc544 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.773435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b414c-d7d8-4f81-9292-79529c7cc544 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.773775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b414c-d7d8-4f81-9292-79529c7cc544 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.774744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b414c-d7d8-4f81-9292-79529c7cc544 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:37.775138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b414c-d7d8-4f81-9292-79529c7cc544 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.776546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b414c-d7d8-4f81-9292-79529c7cc544 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.793133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 402/816] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:37.793944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b414c-d7d8-4f81-9292-79529c7cc544 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.837949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:37.862279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763c210-7699-476b-9a88-807ca6eb68dc None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.894450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b414c-d7d8-4f81-9292-79529c7cc544 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:37.899165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715553f8-8a7e-4753-9c90-57ea6302a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.899769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715553f8-8a7e-4753-9c90-57ea6302a563 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.900051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715553f8-8a7e-4753-9c90-57ea6302a563 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.900321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715553f8-8a7e-4753-9c90-57ea6302a563 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.900941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715553f8-8a7e-4753-9c90-57ea6302a563 None admin] RBAC: Adding query filter params (name=network) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:37.901240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715553f8-8a7e-4753-9c90-57ea6302a563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.902630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715553f8-8a7e-4753-9c90-57ea6302a563 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.907699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 415/817] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:37.909042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715553f8-8a7e-4753-9c90-57ea6302a563 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.929154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1f691-5663-40e6-9831-e670091c2702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:37.929810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1f691-5663-40e6-9831-e670091c2702 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:37.930092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1f691-5663-40e6-9831-e670091c2702 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:37.930367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1f691-5663-40e6-9831-e670091c2702 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:37.936984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1f691-5663-40e6-9831-e670091c2702 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:37.940986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1f691-5663-40e6-9831-e670091c2702 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-b9d1f691-5663-40e6-9831-e670091c2702 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:37.951975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:37.953046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 403/818] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:37.954004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1f691-5663-40e6-9831-e670091c2702 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:37.990296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:38.011558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715553f8-8a7e-4753-9c90-57ea6302a563 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.044711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8aa9-55d7-4f35-a1de-57d1007e0217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.045317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8aa9-55d7-4f35-a1de-57d1007e0217 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.045627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8aa9-55d7-4f35-a1de-57d1007e0217 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.045901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8aa9-55d7-4f35-a1de-57d1007e0217 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.046522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e8aa9-55d7-4f35-a1de-57d1007e0217 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:38.046822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e8aa9-55d7-4f35-a1de-57d1007e0217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.048798 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e8aa9-55d7-4f35-a1de-57d1007e0217 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.051646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1f691-5663-40e6-9831-e670091c2702 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.053054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 416/819] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:38.053159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:38.053974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8aa9-55d7-4f35-a1de-57d1007e0217 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.088823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8103d5-2bc0-4308-b9d0-97f4310568f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.089416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8103d5-2bc0-4308-b9d0-97f4310568f7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.089720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8103d5-2bc0-4308-b9d0-97f4310568f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.089988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8103d5-2bc0-4308-b9d0-97f4310568f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.090666 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8103d5-2bc0-4308-b9d0-97f4310568f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:38.090966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8103d5-2bc0-4308-b9d0-97f4310568f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.092386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8103d5-2bc0-4308-b9d0-97f4310568f7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.104400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 404/820] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:38.105214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8103d5-2bc0-4308-b9d0-97f4310568f7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.150658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8aa9-55d7-4f35-a1de-57d1007e0217 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.183552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f4614-6b63-459e-9ba8-aa00f03de80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.184197 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f4614-6b63-459e-9ba8-aa00f03de80c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.184478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f4614-6b63-459e-9ba8-aa00f03de80c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.184747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f4614-6b63-459e-9ba8-aa00f03de80c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.195869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f4614-6b63-459e-9ba8-aa00f03de80c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.197281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f4614-6b63-459e-9ba8-aa00f03de80c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.198980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:38.199770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8103d5-2bc0-4308-b9d0-97f4310568f7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-810f4614-6b63-459e-9ba8-aa00f03de80c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:38.211935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:38.212862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:38.213772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:38.213772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:38.213772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:38.213772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 417/821] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:38.214015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f4614-6b63-459e-9ba8-aa00f03de80c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.238156 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a38f8-6856-48a9-b9bd-d2d7eab87f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.238773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a38f8-6856-48a9-b9bd-d2d7eab87f7c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.239037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a38f8-6856-48a9-b9bd-d2d7eab87f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.239315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a38f8-6856-48a9-b9bd-d2d7eab87f7c None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.239942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a38f8-6856-48a9-b9bd-d2d7eab87f7c None admin] RBAC: Adding query filter params (type=network) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:38.240250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a38f8-6856-48a9-b9bd-d2d7eab87f7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.241700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a38f8-6856-48a9-b9bd-d2d7eab87f7c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.246229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 405/822] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:09:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:38.246991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a38f8-6856-48a9-b9bd-d2d7eab87f7c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.274039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:38.316230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f4614-6b63-459e-9ba8-aa00f03de80c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.344198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a38f8-6856-48a9-b9bd-d2d7eab87f7c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.349953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e168-0990-485f-a86f-16bf754ee117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.350550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e168-0990-485f-a86f-16bf754ee117 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.350828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e168-0990-485f-a86f-16bf754ee117 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.351100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63e168-0990-485f-a86f-16bf754ee117 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.351783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e168-0990-485f-a86f-16bf754ee117 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:38.352084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e168-0990-485f-a86f-16bf754ee117 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.353538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63e168-0990-485f-a86f-16bf754ee117 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.365626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 418/823] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:38.366408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e168-0990-485f-a86f-16bf754ee117 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.382713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1ebda-db95-40b8-b675-a931f14ecb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.383313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1ebda-db95-40b8-b675-a931f14ecb6b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.383586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1ebda-db95-40b8-b675-a931f14ecb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.383853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1ebda-db95-40b8-b675-a931f14ecb6b None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.384465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1ebda-db95-40b8-b675-a931f14ecb6b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:38.384764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1ebda-db95-40b8-b675-a931f14ecb6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.386147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1ebda-db95-40b8-b675-a931f14ecb6b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.391229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 406/824] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 20:09:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:38.392374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1ebda-db95-40b8-b675-a931f14ecb6b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.420245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:38.464886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63e168-0990-485f-a86f-16bf754ee117 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.479169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:38.490768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1ebda-db95-40b8-b675-a931f14ecb6b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.498178 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573d577-61ee-4d09-a1e1-8f1ba2726d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.498716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573d577-61ee-4d09-a1e1-8f1ba2726d78 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.498999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573d577-61ee-4d09-a1e1-8f1ba2726d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.499268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573d577-61ee-4d09-a1e1-8f1ba2726d78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.499984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573d577-61ee-4d09-a1e1-8f1ba2726d78 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=286f4e203e7a4a55bca0a58d061af2bd) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:38.500288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573d577-61ee-4d09-a1e1-8f1ba2726d78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.502295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573d577-61ee-4d09-a1e1-8f1ba2726d78 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.507839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 419/825] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:38 2022] GET /identity/v3/role_assignments?user.id=286f4e203e7a4a55bca0a58d061af2bd&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=36331ec562ff42f883fc883363bc4c8d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:38.508596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573d577-61ee-4d09-a1e1-8f1ba2726d78 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.533971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c50d08-357a-46b0-a579-17c90a61179e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.534594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c50d08-357a-46b0-a579-17c90a61179e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.535116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c50d08-357a-46b0-a579-17c90a61179e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.535222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c50d08-357a-46b0-a579-17c90a61179e None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.535816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c50d08-357a-46b0-a579-17c90a61179e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:38.536122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c50d08-357a-46b0-a579-17c90a61179e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.537617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c50d08-357a-46b0-a579-17c90a61179e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.550585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 407/826] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:38.551436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c50d08-357a-46b0-a579-17c90a61179e None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.627551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573d577-61ee-4d09-a1e1-8f1ba2726d78 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.647611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:38.662349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c50d08-357a-46b0-a579-17c90a61179e None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.669544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dc3de-f09c-4e05-96b0-a9b46a792627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.670303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dc3de-f09c-4e05-96b0-a9b46a792627 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.670581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dc3de-f09c-4e05-96b0-a9b46a792627 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.670854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dc3de-f09c-4e05-96b0-a9b46a792627 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.671525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431dc3de-f09c-4e05-96b0-a9b46a792627 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.673040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431dc3de-f09c-4e05-96b0-a9b46a792627 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.703126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 420/827] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:09:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:38.705117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dc3de-f09c-4e05-96b0-a9b46a792627 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.721448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d00cf-fb59-4cbc-b522-0117e9b6a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.722379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d00cf-fb59-4cbc-b522-0117e9b6a917 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.723929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d00cf-fb59-4cbc-b522-0117e9b6a917 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.724246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d00cf-fb59-4cbc-b522-0117e9b6a917 None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/f8dec1dbdcd94316aad25c93c313a5a5/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.726528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d00cf-fb59-4cbc-b522-0117e9b6a917 None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=f8dec1dbdcd94316aad25c93c313a5a5, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.728076 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d00cf-fb59-4cbc-b522-0117e9b6a917 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.747882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 408/828] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:09:38 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/f8dec1dbdcd94316aad25c93c313a5a5/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:09:38.749411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d00cf-fb59-4cbc-b522-0117e9b6a917 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:38.763699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:38.845860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431dc3de-f09c-4e05-96b0-a9b46a792627 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.892239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964d38e-9639-4b90-abac-aaffe9a5ed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.893023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964d38e-9639-4b90-abac-aaffe9a5ed14 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.893397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964d38e-9639-4b90-abac-aaffe9a5ed14 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.893956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964d38e-9639-4b90-abac-aaffe9a5ed14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.895237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964d38e-9639-4b90-abac-aaffe9a5ed14 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.897056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964d38e-9639-4b90-abac-aaffe9a5ed14 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.906146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d00cf-fb59-4cbc-b522-0117e9b6a917 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:38.922031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 421/829] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:09:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:38.961829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c625e4-0640-4ac8-a6ee-6a5c17744180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:38.962770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c625e4-0640-4ac8-a6ee-6a5c17744180 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:38.963059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c625e4-0640-4ac8-a6ee-6a5c17744180 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:38.963335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c625e4-0640-4ac8-a6ee-6a5c17744180 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:38.972479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c625e4-0640-4ac8-a6ee-6a5c17744180 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:38.974291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c625e4-0640-4ac8-a6ee-6a5c17744180 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-c1c625e4-0640-4ac8-a6ee-6a5c17744180 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:38.988465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:38.989504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:38.990440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:38.990440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:38.990440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:38.990440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:38.990440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:38.990440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:38.990440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:38.990440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:38.990440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 409/830] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:38.995807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964d38e-9639-4b90-abac-aaffe9a5ed14 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:39.039824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:39.124352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964d38e-9639-4b90-abac-aaffe9a5ed14 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:39.161646 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76ace4-0f5e-4b97-b464-672245a94824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:39.162628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76ace4-0f5e-4b97-b464-672245a94824 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:39.163118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76ace4-0f5e-4b97-b464-672245a94824 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:39.163602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76ace4-0f5e-4b97-b464-672245a94824 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:39.164308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76ace4-0f5e-4b97-b464-672245a94824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:39.164613 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76ace4-0f5e-4b97-b464-672245a94824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:39.166131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76ace4-0f5e-4b97-b464-672245a94824 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:39.179857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 422/831] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:39.185522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c625e4-0640-4ac8-a6ee-6a5c17744180 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:39.257990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:39.300880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c625e4-0640-4ac8-a6ee-6a5c17744180 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:39.336349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178e6a2-d797-441d-b97c-3e56f34d6f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:39.337007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178e6a2-d797-441d-b97c-3e56f34d6f1e None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:39.337287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178e6a2-d797-441d-b97c-3e56f34d6f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:39.337591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178e6a2-d797-441d-b97c-3e56f34d6f1e None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/2314d9416eea4856aec5ac2a0cdfdfc1/roles/89ae1151daf54ba9b292e812aac0ecc6` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:39.352703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178e6a2-d797-441d-b97c-3e56f34d6f1e None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=2314d9416eea4856aec5ac2a0cdfdfc1, role_id=89ae1151daf54ba9b292e812aac0ecc6)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:39.354184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178e6a2-d797-441d-b97c-3e56f34d6f1e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:39.369993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 410/832] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:09:39 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/2314d9416eea4856aec5ac2a0cdfdfc1/roles/89ae1151daf54ba9b292e812aac0ecc6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:09:40.952112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef372b4c-d79a-468e-a3e6-adcd37a5e98a None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:40.953093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef372b4c-d79a-468e-a3e6-adcd37a5e98a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:40.953644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef372b4c-d79a-468e-a3e6-adcd37a5e98a None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:40.955500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 423/833] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:40.960914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d7d7-2e4e-4686-9da7-d430a1483d38 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:40.961733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d7d7-2e4e-4686-9da7-d430a1483d38 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:40.962210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d7d7-2e4e-4686-9da7-d430a1483d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:40.982220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1062d7d7-2e4e-4686-9da7-d430a1483d38 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:40.983848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1062d7d7-2e4e-4686-9da7-d430a1483d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:41.060380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 411/834] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:41.102128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28b389-88e3-46dd-ab71-dcb054ca6872 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.102464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28b389-88e3-46dd-ab71-dcb054ca6872 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.102656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28b389-88e3-46dd-ab71-dcb054ca6872 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.104358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 424/835] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:41.110888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf229b2-ff7a-4fd7-ad9e-4a0f1c0414be None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.111200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf229b2-ff7a-4fd7-ad9e-4a0f1c0414be None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.111439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf229b2-ff7a-4fd7-ad9e-4a0f1c0414be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.131861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-bcf229b2-ff7a-4fd7-ad9e-4a0f1c0414be None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:41.133013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-bcf229b2-ff7a-4fd7-ad9e-4a0f1c0414be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:41.170524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:41.206099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 412/836] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:41.235801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb25b-60c3-4789-a5ef-20c5676bcb21 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.236081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb25b-60c3-4789-a5ef-20c5676bcb21 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.236343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb25b-60c3-4789-a5ef-20c5676bcb21 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.237987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 425/837] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:41.246763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a87ef2-8ee0-41b4-a37d-5af19c209a97 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.247127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a87ef2-8ee0-41b4-a37d-5af19c209a97 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.247400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a87ef2-8ee0-41b4-a37d-5af19c209a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.265711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e5a87ef2-8ee0-41b4-a37d-5af19c209a97 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:41.267649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e5a87ef2-8ee0-41b4-a37d-5af19c209a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:41.341623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 413/838] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:41.346221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb25b-60c3-4789-a5ef-20c5676bcb21 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:41.446248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:41.458262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb25b-60c3-4789-a5ef-20c5676bcb21 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:41.480593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a87ef2-8ee0-41b4-a37d-5af19c209a97 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:41.498330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850486a-a003-4c34-a31b-9aeb06cfcf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:41.499082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850486a-a003-4c34-a31b-9aeb06cfcf3c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.499362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850486a-a003-4c34-a31b-9aeb06cfcf3c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.499775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850486a-a003-4c34-a31b-9aeb06cfcf3c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.500961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850486a-a003-4c34-a31b-9aeb06cfcf3c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:41.502418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850486a-a003-4c34-a31b-9aeb06cfcf3c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-8850486a-a003-4c34-a31b-9aeb06cfcf3c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:41.512715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:41.513687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:41.515266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 426/839] 10.222.0.23 () {56 vars in 1132 bytes} [Wed Oct 5 20:09:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:41.519470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850486a-a003-4c34-a31b-9aeb06cfcf3c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:41.614920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a87ef2-8ee0-41b4-a37d-5af19c209a97 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:41.616533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:41.630808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850486a-a003-4c34-a31b-9aeb06cfcf3c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:41.650153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353472ad-65c0-4e84-8cc8-e5223605e47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:41.650774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353472ad-65c0-4e84-8cc8-e5223605e47c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.651044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353472ad-65c0-4e84-8cc8-e5223605e47c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.651308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353472ad-65c0-4e84-8cc8-e5223605e47c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.661662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353472ad-65c0-4e84-8cc8-e5223605e47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:41.665166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ba527-64d9-4f70-89c9-bc1896e18e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:41.665795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ba527-64d9-4f70-89c9-bc1896e18e90 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.666071 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ba527-64d9-4f70-89c9-bc1896e18e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.666341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ba527-64d9-4f70-89c9-bc1896e18e90 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-353472ad-65c0-4e84-8cc8-e5223605e47c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:41.666440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:41.669596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:09:41.670671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:41.672313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ba527-64d9-4f70-89c9-bc1896e18e90 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:41.672313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ba527-64d9-4f70-89c9-bc1896e18e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:41.673019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 414/840] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:41.673095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ba527-64d9-4f70-89c9-bc1896e18e90 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:41.673163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2eb69-e60a-41ba-9805-2fe5d6a0ee44 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.673163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2eb69-e60a-41ba-9805-2fe5d6a0ee44 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.673163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2eb69-e60a-41ba-9805-2fe5d6a0ee44 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.673163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 415/841] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:41.673163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2eb69-e60a-41ba-9805-2fe5d6a0ee44 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:41.682930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 427/842] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:41.682930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ba527-64d9-4f70-89c9-bc1896e18e90 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:41.699356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:41.775075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2eb69-e60a-41ba-9805-2fe5d6a0ee44 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:41.776485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ba527-64d9-4f70-89c9-bc1896e18e90 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:41.808488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dcf45-d304-4a17-b7b8-035849c0ba2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:41.809108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dcf45-d304-4a17-b7b8-035849c0ba2d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.809383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dcf45-d304-4a17-b7b8-035849c0ba2d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.809485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79aad43-787d-4e82-834f-72d64bfea8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:41.809684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dcf45-d304-4a17-b7b8-035849c0ba2d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.810065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79aad43-787d-4e82-834f-72d64bfea8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.810254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dcf45-d304-4a17-b7b8-035849c0ba2d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:41.810350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79aad43-787d-4e82-834f-72d64bfea8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.810594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79aad43-787d-4e82-834f-72d64bfea8f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.811237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79aad43-787d-4e82-834f-72d64bfea8f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:41.811532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79aad43-787d-4e82-834f-72d64bfea8f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:41.811622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dcf45-d304-4a17-b7b8-035849c0ba2d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:41.812880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79aad43-787d-4e82-834f-72d64bfea8f6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-482dcf45-d304-4a17-b7b8-035849c0ba2d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:41.820502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:41.821681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 20:09:41.822508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:41.823244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 428/843] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:41.823312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 416/844] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:41.823376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081f85f-760a-419a-beec-dca69de5437b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.823451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dcf45-d304-4a17-b7b8-035849c0ba2d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:41.823520 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081f85f-760a-419a-beec-dca69de5437b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.823520 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081f85f-760a-419a-beec-dca69de5437b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.840945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8081f85f-760a-419a-beec-dca69de5437b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:41.842151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8081f85f-760a-419a-beec-dca69de5437b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:41.853342 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:41.910424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 429/845] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:41.911269 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081f85f-760a-419a-beec-dca69de5437b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:41.922204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:41.933662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dcf45-d304-4a17-b7b8-035849c0ba2d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:41.969771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f108ea-e0fa-42ef-8a07-3b9898146bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:41.970361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f108ea-e0fa-42ef-8a07-3b9898146bcf None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:41.970575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f108ea-e0fa-42ef-8a07-3b9898146bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:41.970843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f108ea-e0fa-42ef-8a07-3b9898146bcf None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:41.971466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f108ea-e0fa-42ef-8a07-3b9898146bcf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:41.971763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f108ea-e0fa-42ef-8a07-3b9898146bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:41.973124 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f108ea-e0fa-42ef-8a07-3b9898146bcf None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:41.978375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 417/846] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:41.979143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f108ea-e0fa-42ef-8a07-3b9898146bcf None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:42.042060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081f85f-760a-419a-beec-dca69de5437b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.096993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e73dc-e98b-49dc-8931-e753b2851fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:42.097963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e73dc-e98b-49dc-8931-e753b2851fdf None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:42.098414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e73dc-e98b-49dc-8931-e753b2851fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:42.098854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e73dc-e98b-49dc-8931-e753b2851fdf None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:42.100298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e73dc-e98b-49dc-8931-e753b2851fdf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:42.100787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e73dc-e98b-49dc-8931-e753b2851fdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:42.102970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:42.103392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e73dc-e98b-49dc-8931-e753b2851fdf None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:42.111777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 430/847] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:42.117741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f108ea-e0fa-42ef-8a07-3b9898146bcf None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.117962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e73dc-e98b-49dc-8931-e753b2851fdf None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:42.162250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a562c-1e3c-4949-95bb-87e17dcf585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:42.164325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a562c-1e3c-4949-95bb-87e17dcf585a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:42.164796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a562c-1e3c-4949-95bb-87e17dcf585a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:42.165256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7a562c-1e3c-4949-95bb-87e17dcf585a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:42.180007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a562c-1e3c-4949-95bb-87e17dcf585a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:42.181460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7a562c-1e3c-4949-95bb-87e17dcf585a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-4e7a562c-1e3c-4949-95bb-87e17dcf585a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:42.197189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:42.198205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:42.199073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:42.199073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:42.199073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:42.199073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 418/848] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:42.199396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a562c-1e3c-4949-95bb-87e17dcf585a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:42.208928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:42.236859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e73dc-e98b-49dc-8931-e753b2851fdf None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.273630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83b359-5899-47c6-9910-8bdc738cf709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:42.274238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83b359-5899-47c6-9910-8bdc738cf709 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:42.274517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83b359-5899-47c6-9910-8bdc738cf709 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:42.274777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83b359-5899-47c6-9910-8bdc738cf709 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:42.275399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83b359-5899-47c6-9910-8bdc738cf709 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:42.275782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83b359-5899-47c6-9910-8bdc738cf709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:42.277416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83b359-5899-47c6-9910-8bdc738cf709 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:42.293908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 431/849] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:42.294714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83b359-5899-47c6-9910-8bdc738cf709 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:42.318822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7a562c-1e3c-4949-95bb-87e17dcf585a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.371232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5385af8-c2b9-4b38-ad6b-aaa730060de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:42.372882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5385af8-c2b9-4b38-ad6b-aaa730060de4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:42.373359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5385af8-c2b9-4b38-ad6b-aaa730060de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:42.374054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5385af8-c2b9-4b38-ad6b-aaa730060de4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:42.390101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5385af8-c2b9-4b38-ad6b-aaa730060de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-e5385af8-c2b9-4b38-ad6b-aaa730060de4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:42.398274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:42.399269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:09:42.400066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:42.408515 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 419/850] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:09:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:42.410932 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5385af8-c2b9-4b38-ad6b-aaa730060de4 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:42.415495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:42.424103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83b359-5899-47c6-9910-8bdc738cf709 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.537709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5385af8-c2b9-4b38-ad6b-aaa730060de4 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.580081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2c692-44b7-4fcb-a24c-ba74201dfa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:42.581226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2c692-44b7-4fcb-a24c-ba74201dfa19 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:42.581563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e5443-7281-4011-9032-7fe9b6cfaf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:42.581955 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2c692-44b7-4fcb-a24c-ba74201dfa19 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:42.582171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e5443-7281-4011-9032-7fe9b6cfaf6c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:42.582442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e5443-7281-4011-9032-7fe9b6cfaf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:42.582706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e5443-7281-4011-9032-7fe9b6cfaf6c None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:42.583005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2c692-44b7-4fcb-a24c-ba74201dfa19 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:42.583386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e5443-7281-4011-9032-7fe9b6cfaf6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:42.583683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e5443-7281-4011-9032-7fe9b6cfaf6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:42.584258 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2c692-44b7-4fcb-a24c-ba74201dfa19 None admin] RBAC: Adding query filter params (name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:42.584745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2c692-44b7-4fcb-a24c-ba74201dfa19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:42.585031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e5443-7281-4011-9032-7fe9b6cfaf6c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:42.586512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2c692-44b7-4fcb-a24c-ba74201dfa19 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:42.593365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:42.601179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 432/852] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:42.601604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 420/852] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:09:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:42.602748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e5443-7281-4011-9032-7fe9b6cfaf6c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:42.606298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2c692-44b7-4fcb-a24c-ba74201dfa19 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:42.673548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:42.715744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2c692-44b7-4fcb-a24c-ba74201dfa19 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.736420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e5443-7281-4011-9032-7fe9b6cfaf6c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.756666 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959126ad-123a-4104-8fdb-d437ddc3f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:42.758028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959126ad-123a-4104-8fdb-d437ddc3f178 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:42.758527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959126ad-123a-4104-8fdb-d437ddc3f178 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:42.759011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959126ad-123a-4104-8fdb-d437ddc3f178 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:42.768307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959126ad-123a-4104-8fdb-d437ddc3f178 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:42.770295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959126ad-123a-4104-8fdb-d437ddc3f178 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:42.777171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd9107-f9f9-4bc2-818e-13315a8c1d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:42.777972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd9107-f9f9-4bc2-818e-13315a8c1d46 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:42.778247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd9107-f9f9-4bc2-818e-13315a8c1d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:42.778507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd9107-f9f9-4bc2-818e-13315a8c1d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-959126ad-123a-4104-8fdb-d437ddc3f178 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:42.785150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:42.786099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 20:09:42.787012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:42.789751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 421/853] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:09:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:42.790740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd9107-f9f9-4bc2-818e-13315a8c1d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:42.792190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd9107-f9f9-4bc2-818e-13315a8c1d46 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:42.795037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959126ad-123a-4104-8fdb-d437ddc3f178 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-b4dd9107-f9f9-4bc2-818e-13315a8c1d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:42.811142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:42.812121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:42.813609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:42.813609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:42.813609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:42.813834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 433/854] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:42.818215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd9107-f9f9-4bc2-818e-13315a8c1d46 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:42.887529 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:42.934614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959126ad-123a-4104-8fdb-d437ddc3f178 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.957083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd9107-f9f9-4bc2-818e-13315a8c1d46 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:42.961356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:42.974266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe2315-a546-476c-abb0-533326c4a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:42.974876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe2315-a546-476c-abb0-533326c4a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:42.975207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe2315-a546-476c-abb0-533326c4a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:42.975540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe2315-a546-476c-abb0-533326c4a2e6 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:42.976329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe2315-a546-476c-abb0-533326c4a2e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:42.976703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe2315-a546-476c-abb0-533326c4a2e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:42.978432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe2315-a546-476c-abb0-533326c4a2e6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:42.997896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 422/855] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 20:09:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:43.002970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe2315-a546-476c-abb0-533326c4a2e6 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:43.006553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba760e-266b-41d9-bb6d-67ac4af2e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:43.007160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba760e-266b-41d9-bb6d-67ac4af2e4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:43.007432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba760e-266b-41d9-bb6d-67ac4af2e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:43.007701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba760e-266b-41d9-bb6d-67ac4af2e4b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:43.008338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba760e-266b-41d9-bb6d-67ac4af2e4b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:43.008636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba760e-266b-41d9-bb6d-67ac4af2e4b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:43.010017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba760e-266b-41d9-bb6d-67ac4af2e4b7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:43.022685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 434/856] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:43.027744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba760e-266b-41d9-bb6d-67ac4af2e4b7 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:43.111976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe2315-a546-476c-abb0-533326c4a2e6 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:43.150087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f6b2d-c296-4836-9ed0-ac18680a66dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:43.151036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f6b2d-c296-4836-9ed0-ac18680a66dc None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:43.151484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba760e-266b-41d9-bb6d-67ac4af2e4b7 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:43.151976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f6b2d-c296-4836-9ed0-ac18680a66dc None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:43.152435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f6b2d-c296-4836-9ed0-ac18680a66dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:43.164484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:43.195055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16ee9e-8e06-444a-92aa-ab61b87c3029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:43.195942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16ee9e-8e06-444a-92aa-ab61b87c3029 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:43.196291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16ee9e-8e06-444a-92aa-ab61b87c3029 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:43.196629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16ee9e-8e06-444a-92aa-ab61b87c3029 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:43.208037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16ee9e-8e06-444a-92aa-ab61b87c3029 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:43.209957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16ee9e-8e06-444a-92aa-ab61b87c3029 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:43.215820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-af16ee9e-8e06-444a-92aa-ab61b87c3029 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:43.223791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:43.224775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:43.225723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 435/857] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:43.234645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16ee9e-8e06-444a-92aa-ab61b87c3029 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:43.249915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f6b2d-c296-4836-9ed0-ac18680a66dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:43.253253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f6b2d-c296-4836-9ed0-ac18680a66dc None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:43.344990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16ee9e-8e06-444a-92aa-ab61b87c3029 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:43.383303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 423/858] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 05 20:09:43.384249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f6b2d-c296-4836-9ed0-ac18680a66dc None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:43.385587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba152a1-4b51-48e0-8891-98cc504ffccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:43.386510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba152a1-4b51-48e0-8891-98cc504ffccf None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:43.387089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba152a1-4b51-48e0-8891-98cc504ffccf None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:43.387362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba152a1-4b51-48e0-8891-98cc504ffccf None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:43.388089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba152a1-4b51-48e0-8891-98cc504ffccf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:43.388393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba152a1-4b51-48e0-8891-98cc504ffccf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:43.389858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba152a1-4b51-48e0-8891-98cc504ffccf None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:43.409945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 436/859] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:43.415084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba152a1-4b51-48e0-8891-98cc504ffccf None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:43.446822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:43.496358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:43.526824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f6b2d-c296-4836-9ed0-ac18680a66dc None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:43.531009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba152a1-4b51-48e0-8891-98cc504ffccf None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:43.572282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb4d30-1b26-4efb-b2d4-1954e5322e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:43.572879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb4d30-1b26-4efb-b2d4-1954e5322e95 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:43.573148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb4d30-1b26-4efb-b2d4-1954e5322e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:43.573409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb4d30-1b26-4efb-b2d4-1954e5322e95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:43.578881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17010d-beca-4d93-8066-344c8dde6ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:43.578881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17010d-beca-4d93-8066-344c8dde6ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:43.579209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17010d-beca-4d93-8066-344c8dde6ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:43.579418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17010d-beca-4d93-8066-344c8dde6ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:43.590494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17010d-beca-4d93-8066-344c8dde6ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:43.591806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17010d-beca-4d93-8066-344c8dde6ce2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-4b17010d-beca-4d93-8066-344c8dde6ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:43.606986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:43.607952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:43.608834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:43.608834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:43.608834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:43.613264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 424/860] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:43.614138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17010d-beca-4d93-8066-344c8dde6ce2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:43.662952 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb4d30-1b26-4efb-b2d4-1954e5322e95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:43.664821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb4d30-1b26-4efb-b2d4-1954e5322e95 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:43.699307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:43.719830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17010d-beca-4d93-8066-344c8dde6ce2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:43.755695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad084ff-fdfb-4f98-9cb2-f1a00b206fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:43.756354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad084ff-fdfb-4f98-9cb2-f1a00b206fbd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:43.756639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad084ff-fdfb-4f98-9cb2-f1a00b206fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:43.756897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad084ff-fdfb-4f98-9cb2-f1a00b206fbd None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:43.757646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad084ff-fdfb-4f98-9cb2-f1a00b206fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:43.757947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad084ff-fdfb-4f98-9cb2-f1a00b206fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:43.759430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad084ff-fdfb-4f98-9cb2-f1a00b206fbd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:43.760743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:43.771941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 425/861] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:43.779119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad084ff-fdfb-4f98-9cb2-f1a00b206fbd None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:43.791306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 437/862] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:09:43.798010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb4d30-1b26-4efb-b2d4-1954e5322e95 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:43.891899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad084ff-fdfb-4f98-9cb2-f1a00b206fbd None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:43.913492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb4d30-1b26-4efb-b2d4-1954e5322e95 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:43.927692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12998d4e-304a-4468-9b1a-26fe0d2ee54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:43.928394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12998d4e-304a-4468-9b1a-26fe0d2ee54d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:43.928738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12998d4e-304a-4468-9b1a-26fe0d2ee54d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:43.929070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12998d4e-304a-4468-9b1a-26fe0d2ee54d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:43.936046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12998d4e-304a-4468-9b1a-26fe0d2ee54d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:43.937610 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12998d4e-304a-4468-9b1a-26fe0d2ee54d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:43.945322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-12998d4e-304a-4468-9b1a-26fe0d2ee54d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:43.950224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:43.951084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:43.951935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:43.951935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:43.951935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:43.951935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:43.951935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:43.951935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:43.951935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:43.951935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:43.955417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a8da3-9b87-48ca-9bbe-393f0151cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:43.955417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a8da3-9b87-48ca-9bbe-393f0151cc2d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:43.955417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a8da3-9b87-48ca-9bbe-393f0151cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:43.955417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a8da3-9b87-48ca-9bbe-393f0151cc2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:43.956081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 426/863] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:43.958099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12998d4e-304a-4468-9b1a-26fe0d2ee54d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:43.966435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a8da3-9b87-48ca-9bbe-393f0151cc2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:43.967935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a8da3-9b87-48ca-9bbe-393f0151cc2d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-380a8da3-9b87-48ca-9bbe-393f0151cc2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:43.985938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:43.988258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:43.990420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:43.990420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:43.990420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:43.990420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 438/864] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:43.998410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a8da3-9b87-48ca-9bbe-393f0151cc2d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:44.025436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:44.081990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12998d4e-304a-4468-9b1a-26fe0d2ee54d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:44.109075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a8da3-9b87-48ca-9bbe-393f0151cc2d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:44.122608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663e2d4-c5dc-48a7-b55a-54e28162a626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:44.123651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663e2d4-c5dc-48a7-b55a-54e28162a626 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.124149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663e2d4-c5dc-48a7-b55a-54e28162a626 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.124692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663e2d4-c5dc-48a7-b55a-54e28162a626 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.125749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663e2d4-c5dc-48a7-b55a-54e28162a626 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:44.126260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663e2d4-c5dc-48a7-b55a-54e28162a626 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:44.127936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663e2d4-c5dc-48a7-b55a-54e28162a626 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:44.149460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b87ead-cda0-4283-89f5-9dadf0f27ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:44.150106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b87ead-cda0-4283-89f5-9dadf0f27ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.150385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b87ead-cda0-4283-89f5-9dadf0f27ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.150660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b87ead-cda0-4283-89f5-9dadf0f27ee4 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.151377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b87ead-cda0-4283-89f5-9dadf0f27ee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:44.151610 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b87ead-cda0-4283-89f5-9dadf0f27ee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:44.152972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b87ead-cda0-4283-89f5-9dadf0f27ee4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:44.158904 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 427/865] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:44.160031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663e2d4-c5dc-48a7-b55a-54e28162a626 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:44.167223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 439/866] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:44.175438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b87ead-cda0-4283-89f5-9dadf0f27ee4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:44.219651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:44.279291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663e2d4-c5dc-48a7-b55a-54e28162a626 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:44.282878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b87ead-cda0-4283-89f5-9dadf0f27ee4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:44.296560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:44.317976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d6ba5-96d8-4005-b735-76080b356b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:44.318577 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6ba5-96d8-4005-b735-76080b356b48 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.318849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6ba5-96d8-4005-b735-76080b356b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.319118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6ba5-96d8-4005-b735-76080b356b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.319858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6ba5-96d8-4005-b735-76080b356b48 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=f8dec1dbdcd94316aad25c93c313a5a5) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:44.319974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310a19d-a106-4502-b06f-a94eb01434ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:44.320165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6ba5-96d8-4005-b735-76080b356b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:44.320576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310a19d-a106-4502-b06f-a94eb01434ca None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.320851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310a19d-a106-4502-b06f-a94eb01434ca None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.321122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310a19d-a106-4502-b06f-a94eb01434ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.321649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6ba5-96d8-4005-b735-76080b356b48 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:44.335813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 428/867] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:44 2022] GET /identity/v3/role_assignments?user.id=f8dec1dbdcd94316aad25c93c313a5a5&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=36331ec562ff42f883fc883363bc4c8d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:44.336032 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310a19d-a106-4502-b06f-a94eb01434ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:44.336137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a5df-9319-4f0f-b556-e000c99a9373 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.336233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310a19d-a106-4502-b06f-a94eb01434ca None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:44.336328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a5df-9319-4f0f-b556-e000c99a9373 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.336328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a5df-9319-4f0f-b556-e000c99a9373 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.336328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 429/868] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:44.336978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afd991-1c49-492b-bc91-979894270750 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.337259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afd991-1c49-492b-bc91-979894270750 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.337554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10afd991-1c49-492b-bc91-979894270750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-0310a19d-a106-4502-b06f-a94eb01434ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:44.354546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:44.355635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:44.356621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:44.356621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:44.356621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:44.356621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:44.356621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:44.356621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:44.356621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:44.356621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:44.361159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 440/869] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:44.362451 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310a19d-a106-4502-b06f-a94eb01434ca None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:44.369077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-10afd991-1c49-492b-bc91-979894270750 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:44.370397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-10afd991-1c49-492b-bc91-979894270750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:44.473761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 430/870] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:44.501569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310a19d-a106-4502-b06f-a94eb01434ca None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:44.509886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:44.540179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7aedd5-30e6-449d-be81-fa3a0ad5cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:44.541172 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7aedd5-30e6-449d-be81-fa3a0ad5cb45 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.541699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7aedd5-30e6-449d-be81-fa3a0ad5cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.542173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7aedd5-30e6-449d-be81-fa3a0ad5cb45 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.543068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7aedd5-30e6-449d-be81-fa3a0ad5cb45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:44.543573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7aedd5-30e6-449d-be81-fa3a0ad5cb45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:44.545333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7aedd5-30e6-449d-be81-fa3a0ad5cb45 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:44.558681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 441/871] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:44.565095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afd991-1c49-492b-bc91-979894270750 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:44.577303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1966cf8-349c-404b-8a4e-3dbdd0090708 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.577614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1966cf8-349c-404b-8a4e-3dbdd0090708 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.577894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1966cf8-349c-404b-8a4e-3dbdd0090708 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.581136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 442/872] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:09:44.585244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535ef4d-6902-4425-9208-feec41ad0436 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.585547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535ef4d-6902-4425-9208-feec41ad0436 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.585839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535ef4d-6902-4425-9208-feec41ad0436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.607384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c535ef4d-6902-4425-9208-feec41ad0436 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:44.608577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c535ef4d-6902-4425-9208-feec41ad0436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:44.677745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10afd991-1c49-492b-bc91-979894270750 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:44.687345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 443/873] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:09:44.687562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:44.716319 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de0d05-0790-4eaa-8825-29bfe7ff9312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:44.717200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de0d05-0790-4eaa-8825-29bfe7ff9312 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.717754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de0d05-0790-4eaa-8825-29bfe7ff9312 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.718188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de0d05-0790-4eaa-8825-29bfe7ff9312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.719130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de0d05-0790-4eaa-8825-29bfe7ff9312 None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=2314d9416eea4856aec5ac2a0cdfdfc1) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:44.719596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de0d05-0790-4eaa-8825-29bfe7ff9312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:44.721187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de0d05-0790-4eaa-8825-29bfe7ff9312 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:44.727834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 431/874] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:44 2022] GET /identity/v3/role_assignments?user.id=2314d9416eea4856aec5ac2a0cdfdfc1&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:44.776942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535ef4d-6902-4425-9208-feec41ad0436 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:44.888945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:44.900574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535ef4d-6902-4425-9208-feec41ad0436 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:44.938359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c4de0-7da0-415d-9d37-ca1e5af1fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:44.938976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c4de0-7da0-415d-9d37-ca1e5af1fb04 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:44.939519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c4de0-7da0-415d-9d37-ca1e5af1fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:44.939792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c4de0-7da0-415d-9d37-ca1e5af1fb04 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:44.940442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c4de0-7da0-415d-9d37-ca1e5af1fb04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:44.941830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c4de0-7da0-415d-9d37-ca1e5af1fb04 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:44.959126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 444/875] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:09:44 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:44.974681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de0d05-0790-4eaa-8825-29bfe7ff9312 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:45.112267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de0d05-0790-4eaa-8825-29bfe7ff9312 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:45.155722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe0f08-6005-402c-88e6-f939fa261cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:45.156327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe0f08-6005-402c-88e6-f939fa261cdc None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:45.157254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe0f08-6005-402c-88e6-f939fa261cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:45.157594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe0f08-6005-402c-88e6-f939fa261cdc None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:45.158213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe0f08-6005-402c-88e6-f939fa261cdc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:45.159601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe0f08-6005-402c-88e6-f939fa261cdc None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:45.175605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 432/876] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:09:44 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:46.590253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6de9e9-87a6-42df-94d3-f912d2c87e71 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:46.590804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6de9e9-87a6-42df-94d3-f912d2c87e71 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:46.590804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6de9e9-87a6-42df-94d3-f912d2c87e71 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:46.592340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 445/877] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:46.597959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04916155-a0f4-4ff7-bd4e-e965180404db None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:46.598350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04916155-a0f4-4ff7-bd4e-e965180404db None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:46.598538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04916155-a0f4-4ff7-bd4e-e965180404db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:46.617299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-04916155-a0f4-4ff7-bd4e-e965180404db None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:46.618512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-04916155-a0f4-4ff7-bd4e-e965180404db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:46.642001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:46.695898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 433/878] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:46.966507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6de9e9-87a6-42df-94d3-f912d2c87e71 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:47.080170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6de9e9-87a6-42df-94d3-f912d2c87e71 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:47.096041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bb7e2-8de9-432d-8d41-718e8e1c5c2e None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.096397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bb7e2-8de9-432d-8d41-718e8e1c5c2e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.096584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bb7e2-8de9-432d-8d41-718e8e1c5c2e None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.098484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 434/879] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:09:47.104241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e88c9-fb52-48eb-b594-31f123623f68 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.104573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e88c9-fb52-48eb-b594-31f123623f68 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.104795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e88c9-fb52-48eb-b594-31f123623f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.120944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031b7d7-cd4c-42ce-a7a2-a49ac1491339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:47.121611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031b7d7-cd4c-42ce-a7a2-a49ac1491339 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.121887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031b7d7-cd4c-42ce-a7a2-a49ac1491339 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.122149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031b7d7-cd4c-42ce-a7a2-a49ac1491339 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.122735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031b7d7-cd4c-42ce-a7a2-a49ac1491339 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:47.124122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031b7d7-cd4c-42ce-a7a2-a49ac1491339 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:47.124318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-3a6e88c9-fb52-48eb-b594-31f123623f68 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:47.125829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3a6e88c9-fb52-48eb-b594-31f123623f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-6031b7d7-cd4c-42ce-a7a2-a49ac1491339 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:47.134326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:47.135428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:47.138426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 446/880] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:47.142033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031b7d7-cd4c-42ce-a7a2-a49ac1491339 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:47.161079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:47.233543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 435/881] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:09:47.235389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d706c-a9e5-4b2f-b7af-8a26d55031a2 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.235668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d706c-a9e5-4b2f-b7af-8a26d55031a2 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.235937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d706c-a9e5-4b2f-b7af-8a26d55031a2 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.238527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 436/882] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:47.244024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b91be4-e46f-4781-9e29-ab84408de96d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.244360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b91be4-e46f-4781-9e29-ab84408de96d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.244574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b91be4-e46f-4781-9e29-ab84408de96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.264219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f2b91be4-e46f-4781-9e29-ab84408de96d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:47.265499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f2b91be4-e46f-4781-9e29-ab84408de96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:47.270464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031b7d7-cd4c-42ce-a7a2-a49ac1491339 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:47.307130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e0bae-7182-46d8-9b21-8b0762d153c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:47.307731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e0bae-7182-46d8-9b21-8b0762d153c9 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.308002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e0bae-7182-46d8-9b21-8b0762d153c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.308267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e0bae-7182-46d8-9b21-8b0762d153c9 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.308888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e0bae-7182-46d8-9b21-8b0762d153c9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:47.309234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e0bae-7182-46d8-9b21-8b0762d153c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:47.310620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e0bae-7182-46d8-9b21-8b0762d153c9 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:47.316633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 447/883] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:09:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:47.322192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e0bae-7182-46d8-9b21-8b0762d153c9 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:47.341886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 437/884] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:09:47.356270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500eaad-d263-4de6-a629-d937d32df507 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.356559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500eaad-d263-4de6-a629-d937d32df507 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.356851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500eaad-d263-4de6-a629-d937d32df507 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.358437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 438/885] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:09:47.365584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a9363-e4f4-4bd5-a51b-97228ca29941 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.365940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a9363-e4f4-4bd5-a51b-97228ca29941 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.366281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a9363-e4f4-4bd5-a51b-97228ca29941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.385974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-0d2a9363-e4f4-4bd5-a51b-97228ca29941 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:47.387407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0d2a9363-e4f4-4bd5-a51b-97228ca29941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:47.398009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:47.412985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:47.448600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e0bae-7182-46d8-9b21-8b0762d153c9 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:47.473661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 439/886] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:47.492243 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b5204-9455-4f0d-a3ba-e2e11432fb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:47.492943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b5204-9455-4f0d-a3ba-e2e11432fb59 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.493280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b5204-9455-4f0d-a3ba-e2e11432fb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.493581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b5204-9455-4f0d-a3ba-e2e11432fb59 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.494252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b5204-9455-4f0d-a3ba-e2e11432fb59 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:47.494549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b5204-9455-4f0d-a3ba-e2e11432fb59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:47.496001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b5204-9455-4f0d-a3ba-e2e11432fb59 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:47.502880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 448/887] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:09:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:47.517141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a9363-e4f4-4bd5-a51b-97228ca29941 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:47.625427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b5204-9455-4f0d-a3ba-e2e11432fb59 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:47.642979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a9363-e4f4-4bd5-a51b-97228ca29941 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:47.693843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf622c-dd5d-40fe-a7ce-bdfc12f97691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:47.694799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf622c-dd5d-40fe-a7ce-bdfc12f97691 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.695370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf622c-dd5d-40fe-a7ce-bdfc12f97691 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.695791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf622c-dd5d-40fe-a7ce-bdfc12f97691 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.716820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf622c-dd5d-40fe-a7ce-bdfc12f97691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-27cf622c-dd5d-40fe-a7ce-bdfc12f97691 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:47.722238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:47.723336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:09:47.724344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:47.726830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 440/888] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 20:09:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:47.732694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf622c-dd5d-40fe-a7ce-bdfc12f97691 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:47.736968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:47.742525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b5204-9455-4f0d-a3ba-e2e11432fb59 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:47.779995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4e2c-8864-44b8-8d2a-8c4b632846d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:47.780631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4e2c-8864-44b8-8d2a-8c4b632846d3 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.780909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4e2c-8864-44b8-8d2a-8c4b632846d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.781189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4e2c-8864-44b8-8d2a-8c4b632846d3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.781843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b4e2c-8864-44b8-8d2a-8c4b632846d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:47.783214 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b4e2c-8864-44b8-8d2a-8c4b632846d3 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-873b4e2c-8864-44b8-8d2a-8c4b632846d3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:47.792069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:47.792980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:47.793881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 449/889] 10.222.0.23 () {56 vars in 1132 bytes} [Wed Oct 5 20:09:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:47.794805 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4e2c-8864-44b8-8d2a-8c4b632846d3 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:47.804626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:47.847414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf622c-dd5d-40fe-a7ce-bdfc12f97691 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:47.890031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bd080-5e8a-4a44-93e4-b9b73b0e7f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:47.890647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bd080-5e8a-4a44-93e4-b9b73b0e7f55 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.891512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bd080-5e8a-4a44-93e4-b9b73b0e7f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.891794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bd080-5e8a-4a44-93e4-b9b73b0e7f55 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.892468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bd080-5e8a-4a44-93e4-b9b73b0e7f55 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:47.892772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bd080-5e8a-4a44-93e4-b9b73b0e7f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:47.894290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bd080-5e8a-4a44-93e4-b9b73b0e7f55 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:47.902618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 441/890] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 20:09:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:47.903275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bd080-5e8a-4a44-93e4-b9b73b0e7f55 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:47.909168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4e2c-8864-44b8-8d2a-8c4b632846d3 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:47.945470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed5d2f-5e1d-4040-bd49-2aa01f963c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:47.946127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed5d2f-5e1d-4040-bd49-2aa01f963c06 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:47.946406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed5d2f-5e1d-4040-bd49-2aa01f963c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:47.946671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed5d2f-5e1d-4040-bd49-2aa01f963c06 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:47.947244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed5d2f-5e1d-4040-bd49-2aa01f963c06 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:47.948583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed5d2f-5e1d-4040-bd49-2aa01f963c06 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-3aed5d2f-5e1d-4040-bd49-2aa01f963c06 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:47.959089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:47.960048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:47.960866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 450/891] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:47.961652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed5d2f-5e1d-4040-bd49-2aa01f963c06 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:47.980193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:48.020976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bd080-5e8a-4a44-93e4-b9b73b0e7f55 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.041833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:48.058778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5059ac-3e25-4995-9be1-a1fc91505402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.059445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5059ac-3e25-4995-9be1-a1fc91505402 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.059731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5059ac-3e25-4995-9be1-a1fc91505402 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.060004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5059ac-3e25-4995-9be1-a1fc91505402 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.060633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5059ac-3e25-4995-9be1-a1fc91505402 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:48.060932 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5059ac-3e25-4995-9be1-a1fc91505402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.062329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5059ac-3e25-4995-9be1-a1fc91505402 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.067954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 442/892] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:48.068564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5059ac-3e25-4995-9be1-a1fc91505402 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.081500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed5d2f-5e1d-4040-bd49-2aa01f963c06 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.117603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635090d-c4a9-4a66-9cee-511a56df0bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.118230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635090d-c4a9-4a66-9cee-511a56df0bff None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.118509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635090d-c4a9-4a66-9cee-511a56df0bff None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.118785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635090d-c4a9-4a66-9cee-511a56df0bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.131648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635090d-c4a9-4a66-9cee-511a56df0bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.133090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635090d-c4a9-4a66-9cee-511a56df0bff None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-0635090d-c4a9-4a66-9cee-511a56df0bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:48.147776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:48.148729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:48.149642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:48.149642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:48.149642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:48.149642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 451/893] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:48.149642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635090d-c4a9-4a66-9cee-511a56df0bff None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.185818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5059ac-3e25-4995-9be1-a1fc91505402 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.218134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:48.225380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64a70-3301-4108-808f-f4348689a1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.226138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64a70-3301-4108-808f-f4348689a1bb None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.226416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64a70-3301-4108-808f-f4348689a1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.226690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64a70-3301-4108-808f-f4348689a1bb None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.227321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64a70-3301-4108-808f-f4348689a1bb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:48.227622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64a70-3301-4108-808f-f4348689a1bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.228997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64a70-3301-4108-808f-f4348689a1bb None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.234212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 443/894] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:48.234861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64a70-3301-4108-808f-f4348689a1bb None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.262562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635090d-c4a9-4a66-9cee-511a56df0bff None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.288208 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:48.301448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b25f6-a438-4827-921e-a70e06feb4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.302087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b25f6-a438-4827-921e-a70e06feb4dc None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.302568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b25f6-a438-4827-921e-a70e06feb4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.302839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b25f6-a438-4827-921e-a70e06feb4dc None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.303932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b25f6-a438-4827-921e-a70e06feb4dc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:48.304234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b25f6-a438-4827-921e-a70e06feb4dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.305600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b25f6-a438-4827-921e-a70e06feb4dc None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.317547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 452/895] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:48.317547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b25f6-a438-4827-921e-a70e06feb4dc None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.343734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64a70-3301-4108-808f-f4348689a1bb None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.379366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b422d-c999-4e5f-8188-ba84b1af8323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.379987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b422d-c999-4e5f-8188-ba84b1af8323 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.380498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b422d-c999-4e5f-8188-ba84b1af8323 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.380621 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b422d-c999-4e5f-8188-ba84b1af8323 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.381177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b422d-c999-4e5f-8188-ba84b1af8323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:48.381483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b422d-c999-4e5f-8188-ba84b1af8323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.382941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b422d-c999-4e5f-8188-ba84b1af8323 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.396053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 444/896] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:48.397023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b422d-c999-4e5f-8188-ba84b1af8323 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.431989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b25f6-a438-4827-921e-a70e06feb4dc None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.453938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:48.465791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2db35-728b-47c1-8078-2e0668980f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.466399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2db35-728b-47c1-8078-2e0668980f84 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.466699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2db35-728b-47c1-8078-2e0668980f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.466934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2db35-728b-47c1-8078-2e0668980f84 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.467557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2db35-728b-47c1-8078-2e0668980f84 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:48.467850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2db35-728b-47c1-8078-2e0668980f84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.469790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2db35-728b-47c1-8078-2e0668980f84 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.479545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 453/897] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:48.479545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2db35-728b-47c1-8078-2e0668980f84 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.506336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b422d-c999-4e5f-8188-ba84b1af8323 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.527913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:48.543871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c65398-f33f-404b-b617-fac3bcd5dfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.544477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c65398-f33f-404b-b617-fac3bcd5dfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.544751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c65398-f33f-404b-b617-fac3bcd5dfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.545021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c65398-f33f-404b-b617-fac3bcd5dfd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.545734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c65398-f33f-404b-b617-fac3bcd5dfd5 None admin] RBAC: Adding query filter params (interface=public, service_id=9e49125a55bf441a93e2322c4dd5c47e, region_id=RegionOne) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:48.546038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c65398-f33f-404b-b617-fac3bcd5dfd5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.547989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c65398-f33f-404b-b617-fac3bcd5dfd5 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.557422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 445/898] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/endpoints?service_id=9e49125a55bf441a93e2322c4dd5c47e&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:48.557422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c65398-f33f-404b-b617-fac3bcd5dfd5 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.588244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2db35-728b-47c1-8078-2e0668980f84 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.624185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f1506-b763-47b7-852a-44a0de2afa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.624809 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f1506-b763-47b7-852a-44a0de2afa71 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.625608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f1506-b763-47b7-852a-44a0de2afa71 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.626152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f1506-b763-47b7-852a-44a0de2afa71 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.634400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f1506-b763-47b7-852a-44a0de2afa71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.636170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f1506-b763-47b7-852a-44a0de2afa71 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-897f1506-b763-47b7-852a-44a0de2afa71 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:48.648092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:48.649221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 454/899] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:48.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f1506-b763-47b7-852a-44a0de2afa71 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.662401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c65398-f33f-404b-b617-fac3bcd5dfd5 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.679205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:48.697048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235318ba-fcf0-4002-8f79-907f21e2b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.697607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235318ba-fcf0-4002-8f79-907f21e2b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.697883 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235318ba-fcf0-4002-8f79-907f21e2b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.698150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235318ba-fcf0-4002-8f79-907f21e2b3c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.698840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235318ba-fcf0-4002-8f79-907f21e2b3c1 None admin] RBAC: Adding query filter params (interface=public, service_id=440ea0906a604b3485e13a49ed5e5a58, region_id=RegionOne) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:48.699143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235318ba-fcf0-4002-8f79-907f21e2b3c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.700490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235318ba-fcf0-4002-8f79-907f21e2b3c1 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.707828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 446/900] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/endpoints?service_id=440ea0906a604b3485e13a49ed5e5a58&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:48.709551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235318ba-fcf0-4002-8f79-907f21e2b3c1 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.767353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f1506-b763-47b7-852a-44a0de2afa71 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.778307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:48.808994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8e86c-3a02-4dcc-8f93-381fa35e9f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.809921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8e86c-3a02-4dcc-8f93-381fa35e9f96 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.810360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8e86c-3a02-4dcc-8f93-381fa35e9f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.810771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8e86c-3a02-4dcc-8f93-381fa35e9f96 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.811515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8e86c-3a02-4dcc-8f93-381fa35e9f96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.813083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8e86c-3a02-4dcc-8f93-381fa35e9f96 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.819393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 455/901] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:09:48.820239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8e86c-3a02-4dcc-8f93-381fa35e9f96 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.820600 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235318ba-fcf0-4002-8f79-907f21e2b3c1 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.876712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e722cda-2fe0-43a4-9046-d9c84ba31ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:48.878110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e722cda-2fe0-43a4-9046-d9c84ba31ebd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:48.878591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e722cda-2fe0-43a4-9046-d9c84ba31ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:48.879083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e722cda-2fe0-43a4-9046-d9c84ba31ebd None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:48.880018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e722cda-2fe0-43a4-9046-d9c84ba31ebd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:48.880468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e722cda-2fe0-43a4-9046-d9c84ba31ebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:48.882124 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e722cda-2fe0-43a4-9046-d9c84ba31ebd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:48.910030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 447/902] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:48.911934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e722cda-2fe0-43a4-9046-d9c84ba31ebd None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:48.973213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8e86c-3a02-4dcc-8f93-381fa35e9f96 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:48.973854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:49.020812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df220f6-ba47-42e0-94d0-527c03ac863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:49.021645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df220f6-ba47-42e0-94d0-527c03ac863b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:49.022036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df220f6-ba47-42e0-94d0-527c03ac863b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:49.022392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df220f6-ba47-42e0-94d0-527c03ac863b None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:49.023518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df220f6-ba47-42e0-94d0-527c03ac863b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:49.024900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df220f6-ba47-42e0-94d0-527c03ac863b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:49.031529 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 456/903] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/services => generated 1268 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:09:49.054914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e722cda-2fe0-43a4-9046-d9c84ba31ebd None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:49.095749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8020b9-7493-4b2c-b42e-b6c96316cd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:49.096383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8020b9-7493-4b2c-b42e-b6c96316cd3c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:49.096664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8020b9-7493-4b2c-b42e-b6c96316cd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:49.097225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8020b9-7493-4b2c-b42e-b6c96316cd3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:49.175389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8020b9-7493-4b2c-b42e-b6c96316cd3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:49.177198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8020b9-7493-4b2c-b42e-b6c96316cd3c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:49.221722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:49.284542 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 448/904] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:48 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 05 20:09:49.290124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df220f6-ba47-42e0-94d0-527c03ac863b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:49.339038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:49.399519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df220f6-ba47-42e0-94d0-527c03ac863b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:49.438767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d4ee-6dca-4110-830d-a9f97b9976b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:49.439648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d4ee-6dca-4110-830d-a9f97b9976b3 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:49.439921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d4ee-6dca-4110-830d-a9f97b9976b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:49.440188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d4ee-6dca-4110-830d-a9f97b9976b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:49.451230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97d4ee-6dca-4110-830d-a9f97b9976b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:49.452675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97d4ee-6dca-4110-830d-a9f97b9976b3 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-cf97d4ee-6dca-4110-830d-a9f97b9976b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:49.467522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:49.468463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:49.469391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:49.469391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:49.469391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:49.469391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 457/905] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:49.475691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8020b9-7493-4b2c-b42e-b6c96316cd3c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:49.590233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8020b9-7493-4b2c-b42e-b6c96316cd3c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:49.627165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e35865-1656-4a3c-87ea-97c3b99f91d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:49.627774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e35865-1656-4a3c-87ea-97c3b99f91d3 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:49.628105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e35865-1656-4a3c-87ea-97c3b99f91d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:49.628375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e35865-1656-4a3c-87ea-97c3b99f91d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:49.629012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e35865-1656-4a3c-87ea-97c3b99f91d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:49.629363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e35865-1656-4a3c-87ea-97c3b99f91d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:49.630767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e35865-1656-4a3c-87ea-97c3b99f91d3 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:49.639550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:49.644151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 449/906] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:49.649613 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d4ee-6dca-4110-830d-a9f97b9976b3 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:49.682940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782d999-28de-49cd-a535-2c6664a21642 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:49.683492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782d999-28de-49cd-a535-2c6664a21642 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:49.683492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782d999-28de-49cd-a535-2c6664a21642 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:49.684824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 450/907] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:49.690245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f20a4-d964-46ea-923c-2a5c62d669c3 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:49.690524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f20a4-d964-46ea-923c-2a5c62d669c3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:49.690793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f20a4-d964-46ea-923c-2a5c62d669c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:49.714388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-508f20a4-d964-46ea-923c-2a5c62d669c3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:49.716054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-508f20a4-d964-46ea-923c-2a5c62d669c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:49.749875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:49.762077 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d4ee-6dca-4110-830d-a9f97b9976b3 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:49.800578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 451/908] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:09:49.809138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa323a51-04cb-46ae-b57c-a257785fe94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:49.809138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa323a51-04cb-46ae-b57c-a257785fe94f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:49.809138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa323a51-04cb-46ae-b57c-a257785fe94f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:49.809138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa323a51-04cb-46ae-b57c-a257785fe94f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:49.815075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa323a51-04cb-46ae-b57c-a257785fe94f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:49.816632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa323a51-04cb-46ae-b57c-a257785fe94f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-fa323a51-04cb-46ae-b57c-a257785fe94f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:49.830250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:49.831199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:49.832075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:49.832075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:49.832075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:49.832075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:49.832075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:49.832075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:49.832075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:49.832075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:49.832075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 458/909] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:49.839005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f20a4-d964-46ea-923c-2a5c62d669c3 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:49.951493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f20a4-d964-46ea-923c-2a5c62d669c3 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:49.981090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:49.991436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c254a-27bd-4c78-9239-ec4c2b151928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:49.992043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c254a-27bd-4c78-9239-ec4c2b151928 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:49.992320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c254a-27bd-4c78-9239-ec4c2b151928 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:49.992590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c254a-27bd-4c78-9239-ec4c2b151928 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:49.993334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c254a-27bd-4c78-9239-ec4c2b151928 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:49.993669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c254a-27bd-4c78-9239-ec4c2b151928 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:49.995103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c254a-27bd-4c78-9239-ec4c2b151928 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:50.008072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 452/910] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:50.014617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa323a51-04cb-46ae-b57c-a257785fe94f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:50.068380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c254a-27bd-4c78-9239-ec4c2b151928 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:50.132264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa323a51-04cb-46ae-b57c-a257785fe94f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:50.175554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d44d29-b44b-43dc-bcad-68c240fea575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:50.176159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d44d29-b44b-43dc-bcad-68c240fea575 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:50.176431 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d44d29-b44b-43dc-bcad-68c240fea575 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:50.176721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d44d29-b44b-43dc-bcad-68c240fea575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:50.177553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d44d29-b44b-43dc-bcad-68c240fea575 None admin] RBAC: Adding query filter params (role.id=8d3762a5bc6a44cfb928f7da69fa6388, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=2314d9416eea4856aec5ac2a0cdfdfc1) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:50.177861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d44d29-b44b-43dc-bcad-68c240fea575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:50.179276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d44d29-b44b-43dc-bcad-68c240fea575 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:50.188156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 459/911] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:50 2022] GET /identity/v3/role_assignments?user.id=2314d9416eea4856aec5ac2a0cdfdfc1&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=8d3762a5bc6a44cfb928f7da69fa6388 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:50.190606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c254a-27bd-4c78-9239-ec4c2b151928 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:50.230728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b5d91-18b9-4b69-a837-57fe6adada78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:50.231751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b5d91-18b9-4b69-a837-57fe6adada78 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:50.232222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b5d91-18b9-4b69-a837-57fe6adada78 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:50.232681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b5d91-18b9-4b69-a837-57fe6adada78 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:50.233598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b5d91-18b9-4b69-a837-57fe6adada78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:50.235808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b5d91-18b9-4b69-a837-57fe6adada78 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:50.255561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 453/912] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:09:50 2022] POST /identity/v3/services => generated 241 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:09:51.123203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57a138-063e-4291-8b2e-cd76aafc478e None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:51.124221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57a138-063e-4291-8b2e-cd76aafc478e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:51.124710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57a138-063e-4291-8b2e-cd76aafc478e None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:51.127104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 460/913] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:51.132662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d66858-3aac-41b1-aa56-d93ee8d8b528 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:51.133445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d66858-3aac-41b1-aa56-d93ee8d8b528 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:51.133970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d66858-3aac-41b1-aa56-d93ee8d8b528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:51.152756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-35d66858-3aac-41b1-aa56-d93ee8d8b528 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:51.154395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-35d66858-3aac-41b1-aa56-d93ee8d8b528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:51.204856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:51.237619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 454/914] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:51.384018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb45999-8cce-4112-acd0-95ca4b409638 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:51.384633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb45999-8cce-4112-acd0-95ca4b409638 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:51.385099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb45999-8cce-4112-acd0-95ca4b409638 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:51.386909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 461/915] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:51.395534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65bff9-62c8-46c4-b344-7d633445f5b7 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:51.396031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65bff9-62c8-46c4-b344-7d633445f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:51.396512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65bff9-62c8-46c4-b344-7d633445f5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:51.415494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6d65bff9-62c8-46c4-b344-7d633445f5b7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:51.416965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6d65bff9-62c8-46c4-b344-7d633445f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:51.486703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 455/916] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:51.512926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb45999-8cce-4112-acd0-95ca4b409638 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:51.530438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:51.618262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb45999-8cce-4112-acd0-95ca4b409638 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:51.653142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f403536-e7a5-412c-83b1-2200ab34e4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:51.654143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f403536-e7a5-412c-83b1-2200ab34e4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:51.654615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f403536-e7a5-412c-83b1-2200ab34e4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:51.655093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f403536-e7a5-412c-83b1-2200ab34e4e2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:51.655864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f403536-e7a5-412c-83b1-2200ab34e4e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:51.657456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f403536-e7a5-412c-83b1-2200ab34e4e2 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-5f403536-e7a5-412c-83b1-2200ab34e4e2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:51.666397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:51.667331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 20:09:51.668174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:51.670800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 462/917] 10.222.0.23 () {56 vars in 1132 bytes} [Wed Oct 5 20:09:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:51.679025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65bff9-62c8-46c4-b344-7d633445f5b7 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:51.762067 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f403536-e7a5-412c-83b1-2200ab34e4e2 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:51.780238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65bff9-62c8-46c4-b344-7d633445f5b7 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:51.788396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:51.820002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca953cc-98f0-4ef4-b7a6-2c372c60fe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:51.820613 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca953cc-98f0-4ef4-b7a6-2c372c60fe93 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:51.820892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca953cc-98f0-4ef4-b7a6-2c372c60fe93 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:51.821262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca953cc-98f0-4ef4-b7a6-2c372c60fe93 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:51.822018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca953cc-98f0-4ef4-b7a6-2c372c60fe93 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:51.822322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca953cc-98f0-4ef4-b7a6-2c372c60fe93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:51.823679 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca953cc-98f0-4ef4-b7a6-2c372c60fe93 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:51.830078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 456/918] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:51.835402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca953cc-98f0-4ef4-b7a6-2c372c60fe93 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:51.854374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:51.887574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f403536-e7a5-412c-83b1-2200ab34e4e2 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:51.924506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde95e39-a86d-40e6-ae20-765792e83022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:51.925766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde95e39-a86d-40e6-ae20-765792e83022 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:51.926331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde95e39-a86d-40e6-ae20-765792e83022 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:51.926825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde95e39-a86d-40e6-ae20-765792e83022 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:51.927601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde95e39-a86d-40e6-ae20-765792e83022 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:51.929157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde95e39-a86d-40e6-ae20-765792e83022 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-cde95e39-a86d-40e6-ae20-765792e83022 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:51.939159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:51.940493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 20:09:51.941304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:51.944061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 463/919] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:51.950640 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde95e39-a86d-40e6-ae20-765792e83022 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:51.962465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca953cc-98f0-4ef4-b7a6-2c372c60fe93 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:51.998480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecfc8a-b00b-4a98-9c44-ad8befcc432e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:51.999460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecfc8a-b00b-4a98-9c44-ad8befcc432e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:51.999929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecfc8a-b00b-4a98-9c44-ad8befcc432e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.000940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecfc8a-b00b-4a98-9c44-ad8befcc432e None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:52.001779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecfc8a-b00b-4a98-9c44-ad8befcc432e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:52.002289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecfc8a-b00b-4a98-9c44-ad8befcc432e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:52.003851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecfc8a-b00b-4a98-9c44-ad8befcc432e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:52.009436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 457/920] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:52.014969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecfc8a-b00b-4a98-9c44-ad8befcc432e None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:52.053336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:52.080730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde95e39-a86d-40e6-ae20-765792e83022 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:52.121474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd54921-804b-4583-af01-79a6a00e526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:52.122160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd54921-804b-4583-af01-79a6a00e526b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:52.122381 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd54921-804b-4583-af01-79a6a00e526b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.122651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd54921-804b-4583-af01-79a6a00e526b None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:52.123342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd54921-804b-4583-af01-79a6a00e526b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:52.123877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd54921-804b-4583-af01-79a6a00e526b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:52.124912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd54921-804b-4583-af01-79a6a00e526b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:52.126980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecfc8a-b00b-4a98-9c44-ad8befcc432e None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:52.131979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 464/921] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:52.138493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd54921-804b-4583-af01-79a6a00e526b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:52.149422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:52.167930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c5069-f8a3-4c9c-ace4-53d55ee0052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:52.169295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c5069-f8a3-4c9c-ace4-53d55ee0052d None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:52.169871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c5069-f8a3-4c9c-ace4-53d55ee0052d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.170370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c5069-f8a3-4c9c-ace4-53d55ee0052d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:52.171237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c5069-f8a3-4c9c-ace4-53d55ee0052d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:52.173066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c5069-f8a3-4c9c-ace4-53d55ee0052d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:52.198689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 458/922] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:09:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:52.260067 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd54921-804b-4583-af01-79a6a00e526b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:52.305201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ecd95-d8b2-414f-8bf9-c432f6f655ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:52.306317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ecd95-d8b2-414f-8bf9-c432f6f655ef None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:52.306801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ecd95-d8b2-414f-8bf9-c432f6f655ef None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.307277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ecd95-d8b2-414f-8bf9-c432f6f655ef None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:52.308108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ecd95-d8b2-414f-8bf9-c432f6f655ef None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:52.308630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ecd95-d8b2-414f-8bf9-c432f6f655ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:52.310518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ecd95-d8b2-414f-8bf9-c432f6f655ef None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:52.318257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 465/923] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 20:09:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:52.327014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c5069-f8a3-4c9c-ace4-53d55ee0052d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:52.435960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:52.445282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314ed2-c965-4390-a11a-6628ced9c098 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:52.445599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314ed2-c965-4390-a11a-6628ced9c098 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.445876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314ed2-c965-4390-a11a-6628ced9c098 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:52.447455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 466/924] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:09:52.450038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c5069-f8a3-4c9c-ace4-53d55ee0052d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:52.452739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18effb9-4013-486e-81f4-b2b8f1cb599c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:52.453024 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18effb9-4013-486e-81f4-b2b8f1cb599c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.454084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18effb9-4013-486e-81f4-b2b8f1cb599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:52.473868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f18effb9-4013-486e-81f4-b2b8f1cb599c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:52.475072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f18effb9-4013-486e-81f4-b2b8f1cb599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:52.494671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d99b45-94d4-4809-9d88-ccdac31b4fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:52.495278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d99b45-94d4-4809-9d88-ccdac31b4fad None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:52.495551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d99b45-94d4-4809-9d88-ccdac31b4fad None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.495815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d99b45-94d4-4809-9d88-ccdac31b4fad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:52.496472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d99b45-94d4-4809-9d88-ccdac31b4fad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:52.497870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d99b45-94d4-4809-9d88-ccdac31b4fad None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:52.520948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 459/925] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:09:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:52.522988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a99caf-5aac-41a2-829e-f3d44a5482a4 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:52.523796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a99caf-5aac-41a2-829e-f3d44a5482a4 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.524569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a99caf-5aac-41a2-829e-f3d44a5482a4 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:52.526696 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 460/926] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:52.532293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d7d2-10ac-4184-9b58-86962808c497 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:52.532636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d7d2-10ac-4184-9b58-86962808c497 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.533958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d7d2-10ac-4184-9b58-86962808c497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:52.553168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:52.559328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b9c4d7d2-10ac-4184-9b58-86962808c497 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:52.562288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b9c4d7d2-10ac-4184-9b58-86962808c497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:52.573882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 467/927] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:52.642262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 461/928] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:52.848102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18effb9-4013-486e-81f4-b2b8f1cb599c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:52.935049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d7d2-10ac-4184-9b58-86962808c497 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:52.959799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18effb9-4013-486e-81f4-b2b8f1cb599c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:52.995861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76775672-0af3-4c55-93e5-7a2632ae4674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:52.996489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76775672-0af3-4c55-93e5-7a2632ae4674 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:52.996762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76775672-0af3-4c55-93e5-7a2632ae4674 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:52.997100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76775672-0af3-4c55-93e5-7a2632ae4674 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.008891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76775672-0af3-4c55-93e5-7a2632ae4674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-76775672-0af3-4c55-93e5-7a2632ae4674 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:53.014226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:53.015120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:53.016972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 468/929] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 20:09:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:53.022740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76775672-0af3-4c55-93e5-7a2632ae4674 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:53.033498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:53.060185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d7d2-10ac-4184-9b58-86962808c497 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.080219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:53.098889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b508c9-e561-4478-9c7b-25eafc9bb7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.099798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b508c9-e561-4478-9c7b-25eafc9bb7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.100260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b508c9-e561-4478-9c7b-25eafc9bb7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.100699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b508c9-e561-4478-9c7b-25eafc9bb7e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.101465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b508c9-e561-4478-9c7b-25eafc9bb7e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.103034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b508c9-e561-4478-9c7b-25eafc9bb7e2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-25b508c9-e561-4478-9c7b-25eafc9bb7e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:53.112876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:53.113876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:09:53.114707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:53.116960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 462/930] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:53.122393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b508c9-e561-4478-9c7b-25eafc9bb7e2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:53.135999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76775672-0af3-4c55-93e5-7a2632ae4674 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.178083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc37e5-3726-4b67-88c3-7048b10d202e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.178706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc37e5-3726-4b67-88c3-7048b10d202e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.178986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc37e5-3726-4b67-88c3-7048b10d202e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.179251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc37e5-3726-4b67-88c3-7048b10d202e None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.179931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc37e5-3726-4b67-88c3-7048b10d202e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:53.180231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc37e5-3726-4b67-88c3-7048b10d202e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.181760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc37e5-3726-4b67-88c3-7048b10d202e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:53.191604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 469/931] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 20:09:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:53.198212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc37e5-3726-4b67-88c3-7048b10d202e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:53.252617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b508c9-e561-4478-9c7b-25eafc9bb7e2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.295019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686586f-9ee8-4308-ba98-ef6ec1010696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.295725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686586f-9ee8-4308-ba98-ef6ec1010696 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.296071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686586f-9ee8-4308-ba98-ef6ec1010696 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.296408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686586f-9ee8-4308-ba98-ef6ec1010696 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.297544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686586f-9ee8-4308-ba98-ef6ec1010696 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:53.297941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686586f-9ee8-4308-ba98-ef6ec1010696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.299740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686586f-9ee8-4308-ba98-ef6ec1010696 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:53.310662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 463/932] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:09:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:53.313535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686586f-9ee8-4308-ba98-ef6ec1010696 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:53.320053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:53.329437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc37e5-3726-4b67-88c3-7048b10d202e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.346904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:53.371278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9082c-8fa8-4688-9f11-38a64d6b34c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.371829 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9082c-8fa8-4688-9f11-38a64d6b34c0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.372389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9082c-8fa8-4688-9f11-38a64d6b34c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.373073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9082c-8fa8-4688-9f11-38a64d6b34c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.385558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9082c-8fa8-4688-9f11-38a64d6b34c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.386907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9082c-8fa8-4688-9f11-38a64d6b34c0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-4bd9082c-8fa8-4688-9f11-38a64d6b34c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:53.404199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:53.405174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:53.406079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:53.406079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:53.406079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:53.410693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 470/933] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:53.411799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9082c-8fa8-4688-9f11-38a64d6b34c0 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:53.451034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686586f-9ee8-4308-ba98-ef6ec1010696 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.495364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062c8cd-84ae-4aed-ba04-f431f5513640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.496741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062c8cd-84ae-4aed-ba04-f431f5513640 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.497017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062c8cd-84ae-4aed-ba04-f431f5513640 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.497285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062c8cd-84ae-4aed-ba04-f431f5513640 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.497953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062c8cd-84ae-4aed-ba04-f431f5513640 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:53.498256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062c8cd-84ae-4aed-ba04-f431f5513640 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.499622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062c8cd-84ae-4aed-ba04-f431f5513640 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:53.506583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 464/934] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:09:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:53.511967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062c8cd-84ae-4aed-ba04-f431f5513640 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:53.537527 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9082c-8fa8-4688-9f11-38a64d6b34c0 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.578013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1cc1d-2c61-41df-a92d-488e6449edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.579141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1cc1d-2c61-41df-a92d-488e6449edea None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.579416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1cc1d-2c61-41df-a92d-488e6449edea None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.579711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1cc1d-2c61-41df-a92d-488e6449edea None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.580625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1cc1d-2c61-41df-a92d-488e6449edea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:53.580923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1cc1d-2c61-41df-a92d-488e6449edea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.582711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1cc1d-2c61-41df-a92d-488e6449edea None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:53.588110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:53.605539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 471/935] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:53.606895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1cc1d-2c61-41df-a92d-488e6449edea None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:53.627467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062c8cd-84ae-4aed-ba04-f431f5513640 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.638165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:53.663005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafaa5e-5888-49fe-ad35-fbcfb7dac2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.663544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafaa5e-5888-49fe-ad35-fbcfb7dac2eb None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.663924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafaa5e-5888-49fe-ad35-fbcfb7dac2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.664344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafaa5e-5888-49fe-ad35-fbcfb7dac2eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.665145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafaa5e-5888-49fe-ad35-fbcfb7dac2eb None admin] RBAC: Adding query filter params (interface=public, service_id=48fa3f932a924eda82c18bf548589795, region_id=RegionOne) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:53.665597 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafaa5e-5888-49fe-ad35-fbcfb7dac2eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.667093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafaa5e-5888-49fe-ad35-fbcfb7dac2eb None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:53.677864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 465/936] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 20:09:53 2022] GET /identity/v3/endpoints?service_id=48fa3f932a924eda82c18bf548589795&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:53.678814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafaa5e-5888-49fe-ad35-fbcfb7dac2eb None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:53.723182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1cc1d-2c61-41df-a92d-488e6449edea None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.762593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c5cf6-3d9b-4016-8136-0bd6f942880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.763438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c5cf6-3d9b-4016-8136-0bd6f942880f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.763854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c5cf6-3d9b-4016-8136-0bd6f942880f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.764286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c5cf6-3d9b-4016-8136-0bd6f942880f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.773552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c5cf6-3d9b-4016-8136-0bd6f942880f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.775309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908c5cf6-3d9b-4016-8136-0bd6f942880f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:53.781623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafaa5e-5888-49fe-ad35-fbcfb7dac2eb None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-908c5cf6-3d9b-4016-8136-0bd6f942880f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:53.789166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:53.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:53.791403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:53.798111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 472/937] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:09:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:53.799140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c5cf6-3d9b-4016-8136-0bd6f942880f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:53.823248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:53.831482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac6837-58e4-4c5b-bb23-826a6bacac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.832711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac6837-58e4-4c5b-bb23-826a6bacac80 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.833449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac6837-58e4-4c5b-bb23-826a6bacac80 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.834022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac6837-58e4-4c5b-bb23-826a6bacac80 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.835010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac6837-58e4-4c5b-bb23-826a6bacac80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.837255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac6837-58e4-4c5b-bb23-826a6bacac80 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:53.846137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 466/938] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 20:09:53 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:09:53.918394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:53.940951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c5cf6-3d9b-4016-8136-0bd6f942880f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:53.979906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8882d-4596-4d1b-ab46-a3b3c6aa1951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:53.980532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8882d-4596-4d1b-ab46-a3b3c6aa1951 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:53.980811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8882d-4596-4d1b-ab46-a3b3c6aa1951 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:53.981079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8882d-4596-4d1b-ab46-a3b3c6aa1951 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:53.981858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8882d-4596-4d1b-ab46-a3b3c6aa1951 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:53.982162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8882d-4596-4d1b-ab46-a3b3c6aa1951 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:53.983607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8882d-4596-4d1b-ab46-a3b3c6aa1951 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:54.004615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 473/939] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 20:09:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:54.010453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac6837-58e4-4c5b-bb23-826a6bacac80 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:54.116523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac6837-58e4-4c5b-bb23-826a6bacac80 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:54.150948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789530f-d70f-4e09-9daa-4ceee87995e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:54.151917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789530f-d70f-4e09-9daa-4ceee87995e2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:54.152387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789530f-d70f-4e09-9daa-4ceee87995e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:54.152866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789530f-d70f-4e09-9daa-4ceee87995e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:54.164309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789530f-d70f-4e09-9daa-4ceee87995e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:54.166020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789530f-d70f-4e09-9daa-4ceee87995e2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-a789530f-d70f-4e09-9daa-4ceee87995e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:54.180210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:54.181119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:54.182020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:54.182020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:54.182020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:54.184034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 467/940] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:54.190117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8882d-4596-4d1b-ab46-a3b3c6aa1951 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:54.294703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8882d-4596-4d1b-ab46-a3b3c6aa1951 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:54.330805 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a6f83-2ad3-4b31-af2f-ba106f9d242c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:54.331794 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6f83-2ad3-4b31-af2f-ba106f9d242c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:54.332308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6f83-2ad3-4b31-af2f-ba106f9d242c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:54.332769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6f83-2ad3-4b31-af2f-ba106f9d242c None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:54.333628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a6f83-2ad3-4b31-af2f-ba106f9d242c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:54.334132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a6f83-2ad3-4b31-af2f-ba106f9d242c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:54.335683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a6f83-2ad3-4b31-af2f-ba106f9d242c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:54.348677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 474/941] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:54.354608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789530f-d70f-4e09-9daa-4ceee87995e2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:54.418912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:54.465027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789530f-d70f-4e09-9daa-4ceee87995e2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:54.500339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce76a75-bb93-4f97-8b9b-fcdd9d1573a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:54.500964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce76a75-bb93-4f97-8b9b-fcdd9d1573a1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:54.501240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce76a75-bb93-4f97-8b9b-fcdd9d1573a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:54.501523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce76a75-bb93-4f97-8b9b-fcdd9d1573a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:54.508733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce76a75-bb93-4f97-8b9b-fcdd9d1573a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:54.509095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e158a-92b6-4467-93dc-69e93d849fa8 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:54.509372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e158a-92b6-4467-93dc-69e93d849fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:54.509661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e158a-92b6-4467-93dc-69e93d849fa8 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:54.510255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce76a75-bb93-4f97-8b9b-fcdd9d1573a1 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:54.511429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 475/942] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:54.516985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13c0c3-e6de-40d8-8c3c-a0a6fb0e7457 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:54.517260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13c0c3-e6de-40d8-8c3c-a0a6fb0e7457 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:54.517540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13c0c3-e6de-40d8-8c3c-a0a6fb0e7457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-4ce76a75-bb93-4f97-8b9b-fcdd9d1573a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:54.524278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:54.525151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:54.526075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:54.526075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:54.526075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:54.526075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:54.526075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:54.526075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:54.526075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:54.526075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:54.526493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:54.526562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 468/943] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:54.532465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce76a75-bb93-4f97-8b9b-fcdd9d1573a1 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:54.539753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-bf13c0c3-e6de-40d8-8c3c-a0a6fb0e7457 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:54.541057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-bf13c0c3-e6de-40d8-8c3c-a0a6fb0e7457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:54.621779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 476/944] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:09:54.639046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce76a75-bb93-4f97-8b9b-fcdd9d1573a1 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:54.649925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:54.674576 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c9259-bc6e-4f6f-bd22-654f0e1d63ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:54.675161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c9259-bc6e-4f6f-bd22-654f0e1d63ee None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:54.675490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c9259-bc6e-4f6f-bd22-654f0e1d63ee None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:54.675862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c9259-bc6e-4f6f-bd22-654f0e1d63ee None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:54.676883 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c9259-bc6e-4f6f-bd22-654f0e1d63ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:54.677412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c9259-bc6e-4f6f-bd22-654f0e1d63ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:54.679151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c9259-bc6e-4f6f-bd22-654f0e1d63ee None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:54.692639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 469/945] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:54.698607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13c0c3-e6de-40d8-8c3c-a0a6fb0e7457 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:54.812686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13c0c3-e6de-40d8-8c3c-a0a6fb0e7457 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:54.840860 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515aff-e687-4814-a8af-7dca5056e968 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:54.841193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515aff-e687-4814-a8af-7dca5056e968 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:54.841457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515aff-e687-4814-a8af-7dca5056e968 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:54.842950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 470/946] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:09:54.848947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf1821-4473-4bb2-a2cf-c2b4d29b8481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:54.849692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132e8d-aa1c-4205-8436-a9a26b7354ba None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:54.849800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf1821-4473-4bb2-a2cf-c2b4d29b8481 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:54.849978 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf1821-4473-4bb2-a2cf-c2b4d29b8481 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:54.850237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf1821-4473-4bb2-a2cf-c2b4d29b8481 None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/2314d9416eea4856aec5ac2a0cdfdfc1/roles/8d3762a5bc6a44cfb928f7da69fa6388` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:54.850681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132e8d-aa1c-4205-8436-a9a26b7354ba None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:54.851139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132e8d-aa1c-4205-8436-a9a26b7354ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:54.852396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf1821-4473-4bb2-a2cf-c2b4d29b8481 None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=2314d9416eea4856aec5ac2a0cdfdfc1, role_id=8d3762a5bc6a44cfb928f7da69fa6388)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:54.853846 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf1821-4473-4bb2-a2cf-c2b4d29b8481 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:54.856276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:54.871881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6a132e8d-aa1c-4205-8436-a9a26b7354ba None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:54.873471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6a132e8d-aa1c-4205-8436-a9a26b7354ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:54.880451 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 477/947] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:09:54 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/2314d9416eea4856aec5ac2a0cdfdfc1/roles/8d3762a5bc6a44cfb928f7da69fa6388 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:09:54.918624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf1821-4473-4bb2-a2cf-c2b4d29b8481 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:54.972999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 471/948] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:09:55.040068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf1821-4473-4bb2-a2cf-c2b4d29b8481 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:55.081210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba7d40-a7ce-4e0f-aa46-383e24362064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:55.081868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba7d40-a7ce-4e0f-aa46-383e24362064 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:55.082148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba7d40-a7ce-4e0f-aa46-383e24362064 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:55.082408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba7d40-a7ce-4e0f-aa46-383e24362064 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:55.082978 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba7d40-a7ce-4e0f-aa46-383e24362064 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:55.084317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba7d40-a7ce-4e0f-aa46-383e24362064 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-dfba7d40-a7ce-4e0f-aa46-383e24362064 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:55.093029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:55.094049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:55.094844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 478/949] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:55.100107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a132e8d-aa1c-4205-8436-a9a26b7354ba None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:55.201747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:55.205814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a132e8d-aa1c-4205-8436-a9a26b7354ba None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:55.245855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5fb69-b5dc-4e92-9edb-00a82454c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:55.246387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5fb69-b5dc-4e92-9edb-00a82454c746 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:55.246665 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5fb69-b5dc-4e92-9edb-00a82454c746 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:55.246937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5fb69-b5dc-4e92-9edb-00a82454c746 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:55.247563 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5fb69-b5dc-4e92-9edb-00a82454c746 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:55.247864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5fb69-b5dc-4e92-9edb-00a82454c746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:55.249238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5fb69-b5dc-4e92-9edb-00a82454c746 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:55.254457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 472/950] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:09:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:55.260055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba7d40-a7ce-4e0f-aa46-383e24362064 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:55.301051 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5fb69-b5dc-4e92-9edb-00a82454c746 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:55.322453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:55.369686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba7d40-a7ce-4e0f-aa46-383e24362064 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:55.407679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d1158-b107-481c-9d95-70e99e2f633f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:55.408282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1158-b107-481c-9d95-70e99e2f633f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:55.408860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1158-b107-481c-9d95-70e99e2f633f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:55.409145 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d1158-b107-481c-9d95-70e99e2f633f None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:55.409808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d1158-b107-481c-9d95-70e99e2f633f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:55.410343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d1158-b107-481c-9d95-70e99e2f633f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:55.411711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d1158-b107-481c-9d95-70e99e2f633f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:55.415533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5fb69-b5dc-4e92-9edb-00a82454c746 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:55.421612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 479/951] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:09:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:55.455308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:55.455924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:55.456198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:55.456464 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:55.457116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:55.458669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:55.461545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING keystone.server.flask.common [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 05 20:09:55.486417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.sql.core [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 05 20:09:55.486417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 05 20:09:55.486417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [parameters: {'id': 'c51009c0bb724299bed6a3ac5e604f16', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 05 20:09:55.486417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81338) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c51009c0bb724299bed6a3ac5e604f16.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c51009c0bb724299bed6a3ac5e604f16. Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application conn.query(q) Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 20:09:55.494023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application result.read() Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 05 20:09:55.494887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application next(self.gen) Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application yield resource Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application next(self.gen) Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.session.rollback() Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.force_reraise() Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise self.value Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application session.commit() Oct 05 20:09:55.495812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._prepare_impl() Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.session.flush() Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._flush(objects) Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application compat.raise_( Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application flush_context.execute() Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 05 20:09:55.496730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rec.execute(self) Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 05 20:09:55.497667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application conn.query(q) Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application result.read() Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 20:09:55.498442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [parameters: {'id': 'c51009c0bb724299bed6a3ac5e604f16', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:55.499300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:55.500113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c51009c0bb724299bed6a3ac5e604f16. Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:55.500992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 473/952] 10.222.0.23 () {60 vars in 1140 bytes} [Wed Oct 5 20:09:55 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:55.509415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d1158-b107-481c-9d95-70e99e2f633f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:55.627771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d1158-b107-481c-9d95-70e99e2f633f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:55.652862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:55.667355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2afb8c-1e20-4186-922a-806ebf9fbbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:55.668356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2afb8c-1e20-4186-922a-806ebf9fbbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:55.668838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2afb8c-1e20-4186-922a-806ebf9fbbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:55.669315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2afb8c-1e20-4186-922a-806ebf9fbbb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:55.677118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2afb8c-1e20-4186-922a-806ebf9fbbb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:55.678891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2afb8c-1e20-4186-922a-806ebf9fbbb5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-4e2afb8c-1e20-4186-922a-806ebf9fbbb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:55.692098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:55.693085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:55.694031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:55.694031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:55.694031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:55.694031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:55.694031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:55.694031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:55.694031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:55.694031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:55.696510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 480/953] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:55.702735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:55.724702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:55.815684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630cccf8-b4b5-4f2a-a492-608a8b29d3d7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:55.853035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d0856-8a76-459d-8420-1cba350a50ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:55.854072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d0856-8a76-459d-8420-1cba350a50ef None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:55.854551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d0856-8a76-459d-8420-1cba350a50ef None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:55.855028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d0856-8a76-459d-8420-1cba350a50ef None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:55.855890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d0856-8a76-459d-8420-1cba350a50ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:55.856390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d0856-8a76-459d-8420-1cba350a50ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:55.858321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d0856-8a76-459d-8420-1cba350a50ef None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:55.887829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 474/954] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:09:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:56.092655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309d5ef-f6ff-40b2-a0b6-e452c932893d None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:56.092983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309d5ef-f6ff-40b2-a0b6-e452c932893d None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:56.093208 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309d5ef-f6ff-40b2-a0b6-e452c932893d None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:56.094863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 481/955] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:56.100385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2020f63-0128-4a37-a4f6-3a677d71dccd None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:56.100613 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2020f63-0128-4a37-a4f6-3a677d71dccd None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:56.100884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2020f63-0128-4a37-a4f6-3a677d71dccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:56.119931 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c2020f63-0128-4a37-a4f6-3a677d71dccd None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:56.121443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c2020f63-0128-4a37-a4f6-3a677d71dccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:56.193748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 475/956] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:56.461760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309d5ef-f6ff-40b2-a0b6-e452c932893d None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:56.569231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309d5ef-f6ff-40b2-a0b6-e452c932893d None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:56.603886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73396e-4fd3-4bda-857d-bc2edb0abd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:56.604894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73396e-4fd3-4bda-857d-bc2edb0abd2d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:56.605371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73396e-4fd3-4bda-857d-bc2edb0abd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:56.605880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73396e-4fd3-4bda-857d-bc2edb0abd2d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:56.606689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73396e-4fd3-4bda-857d-bc2edb0abd2d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:56.608234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73396e-4fd3-4bda-857d-bc2edb0abd2d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-4c73396e-4fd3-4bda-857d-bc2edb0abd2d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:56.618144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:56.619200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:09:56.620055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:56.622786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 482/957] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:56.633568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2020f63-0128-4a37-a4f6-3a677d71dccd None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:56.639346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:56.734886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2020f63-0128-4a37-a4f6-3a677d71dccd None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:56.769998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0184dfe-b931-4f5f-84d6-67636de0c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:56.770612 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184dfe-b931-4f5f-84d6-67636de0c4ac None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:56.770885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184dfe-b931-4f5f-84d6-67636de0c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:56.771157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0184dfe-b931-4f5f-84d6-67636de0c4ac None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:56.771772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0184dfe-b931-4f5f-84d6-67636de0c4ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:56.772066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0184dfe-b931-4f5f-84d6-67636de0c4ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:56.773428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0184dfe-b931-4f5f-84d6-67636de0c4ac None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:56.778940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 476/958] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:09:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:56.785103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73396e-4fd3-4bda-857d-bc2edb0abd2d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:56.833533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:56.907571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73396e-4fd3-4bda-857d-bc2edb0abd2d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:56.943801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112192d2-6143-475d-af0b-7c687a575149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:56.944454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112192d2-6143-475d-af0b-7c687a575149 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:56.944733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112192d2-6143-475d-af0b-7c687a575149 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:56.945002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112192d2-6143-475d-af0b-7c687a575149 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:56.945652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112192d2-6143-475d-af0b-7c687a575149 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:56.945950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112192d2-6143-475d-af0b-7c687a575149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:56.947311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112192d2-6143-475d-af0b-7c687a575149 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:56.952516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 483/959] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:09:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:56.958192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0184dfe-b931-4f5f-84d6-67636de0c4ac None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:57.034066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:57.067308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0184dfe-b931-4f5f-84d6-67636de0c4ac None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:57.111367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67bc7b-0f7a-4e1a-903b-46df318ae78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:57.111988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67bc7b-0f7a-4e1a-903b-46df318ae78f None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:57.112269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67bc7b-0f7a-4e1a-903b-46df318ae78f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:57.112546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67bc7b-0f7a-4e1a-903b-46df318ae78f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:57.113245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67bc7b-0f7a-4e1a-903b-46df318ae78f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:57.114667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67bc7b-0f7a-4e1a-903b-46df318ae78f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:57.139712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 477/960] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:09:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:09:57.208271 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b455000-5614-4534-8812-48b0469f505a None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:57.208598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b455000-5614-4534-8812-48b0469f505a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:57.208821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b455000-5614-4534-8812-48b0469f505a None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:57.210831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 484/961] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:57.216654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92679b38-63a9-48cf-863c-16ebd8eccbfe None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:57.216991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92679b38-63a9-48cf-863c-16ebd8eccbfe None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:57.217220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92679b38-63a9-48cf-863c-16ebd8eccbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:57.238493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-92679b38-63a9-48cf-863c-16ebd8eccbfe None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:57.239804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-92679b38-63a9-48cf-863c-16ebd8eccbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:57.322953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 478/962] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:57.591247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b455000-5614-4534-8812-48b0469f505a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:57.704006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b455000-5614-4534-8812-48b0469f505a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:57.715363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:57.735523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3843186-e350-45cb-98a0-f66952f88102 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:57.736039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3843186-e350-45cb-98a0-f66952f88102 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:57.736162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3843186-e350-45cb-98a0-f66952f88102 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:57.738800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 479/963] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:57.742770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2ac8a-bc71-4700-a54e-f71713b99d39 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:57.743054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2ac8a-bc71-4700-a54e-f71713b99d39 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:57.743231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b72d89-1adc-4410-bdc3-883ff6fad127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:57.743363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2ac8a-bc71-4700-a54e-f71713b99d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:57.743898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b72d89-1adc-4410-bdc3-883ff6fad127 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:57.744176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b72d89-1adc-4410-bdc3-883ff6fad127 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:57.744445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b72d89-1adc-4410-bdc3-883ff6fad127 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:57.758732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b72d89-1adc-4410-bdc3-883ff6fad127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:09:57.761485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b6e2ac8a-bc71-4700-a54e-f71713b99d39 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:57.762712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b6e2ac8a-bc71-4700-a54e-f71713b99d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-66b72d89-1adc-4410-bdc3-883ff6fad127 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:09:57.766445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:57.767377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:09:57.768187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:57.770890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 485/964] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 20:09:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:57.776334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b72d89-1adc-4410-bdc3-883ff6fad127 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:57.818652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:57.845702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 480/965] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:09:57.903342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b72d89-1adc-4410-bdc3-883ff6fad127 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:57.939741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d928ba-19b8-41bf-bde1-91e8026183c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:57.940726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d928ba-19b8-41bf-bde1-91e8026183c1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:57.941202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d928ba-19b8-41bf-bde1-91e8026183c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:57.941710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d928ba-19b8-41bf-bde1-91e8026183c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:57.942729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d928ba-19b8-41bf-bde1-91e8026183c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:57.943228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d928ba-19b8-41bf-bde1-91e8026183c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:57.944808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d928ba-19b8-41bf-bde1-91e8026183c1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:57.954679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 486/966] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 20:09:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:57.962913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2ac8a-bc71-4700-a54e-f71713b99d39 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:58.070128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2ac8a-bc71-4700-a54e-f71713b99d39 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:58.107560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68e540-965e-4fda-8db3-caf9d41c89b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:58.108167 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68e540-965e-4fda-8db3-caf9d41c89b4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:58.108449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68e540-965e-4fda-8db3-caf9d41c89b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:58.108717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68e540-965e-4fda-8db3-caf9d41c89b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:58.119197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68e540-965e-4fda-8db3-caf9d41c89b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:58.120657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68e540-965e-4fda-8db3-caf9d41c89b4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:58.132680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d928ba-19b8-41bf-bde1-91e8026183c1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-2b68e540-965e-4fda-8db3-caf9d41c89b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:58.141447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:58.142461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:58.143327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:58.143327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:58.143327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:58.143327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 481/967] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:58.148572 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68e540-965e-4fda-8db3-caf9d41c89b4 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:58.171221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:58.188907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:58.256298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68e540-965e-4fda-8db3-caf9d41c89b4 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:58.258608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d928ba-19b8-41bf-bde1-91e8026183c1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:58.290945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1ec6b-6e03-49c9-8aea-a6787de65039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:58.291547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1ec6b-6e03-49c9-8aea-a6787de65039 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:58.291823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1ec6b-6e03-49c9-8aea-a6787de65039 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:58.292096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1ec6b-6e03-49c9-8aea-a6787de65039 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:58.292726 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1ec6b-6e03-49c9-8aea-a6787de65039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:58.293024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1ec6b-6e03-49c9-8aea-a6787de65039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:58.293551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f120d9d-dd14-4219-b95c-6a2657099cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:58.294156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f120d9d-dd14-4219-b95c-6a2657099cc1 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:58.294433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f120d9d-dd14-4219-b95c-6a2657099cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:58.294527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1ec6b-6e03-49c9-8aea-a6787de65039 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:58.294705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f120d9d-dd14-4219-b95c-6a2657099cc1 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:58.295343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f120d9d-dd14-4219-b95c-6a2657099cc1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:58.296680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f120d9d-dd14-4219-b95c-6a2657099cc1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:58.310979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 482/968] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:58.310979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ec08-8283-48da-a389-567862b27241 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:58.310979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ec08-8283-48da-a389-567862b27241 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:58.310979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ec08-8283-48da-a389-567862b27241 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:58.313632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 483/969] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:09:58.314870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40ec08-8283-48da-a389-567862b27241 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:58.318794 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 487/970] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:09:58 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:09:58.320215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebe970-2fc4-4966-98d3-3e90c2e498c2 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:58.320504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebe970-2fc4-4966-98d3-3e90c2e498c2 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:58.320775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebe970-2fc4-4966-98d3-3e90c2e498c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:58.341184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-08ebe970-2fc4-4966-98d3-3e90c2e498c2 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:58.342489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-08ebe970-2fc4-4966-98d3-3e90c2e498c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:58.427413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 488/971] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:58.440012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:58.447825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40ec08-8283-48da-a389-567862b27241 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:58.495803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c11ad-e59f-49fb-a1b2-07ef33c167e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:58.496517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c11ad-e59f-49fb-a1b2-07ef33c167e9 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:58.496808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c11ad-e59f-49fb-a1b2-07ef33c167e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:58.497066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c11ad-e59f-49fb-a1b2-07ef33c167e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:58.505949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c11ad-e59f-49fb-a1b2-07ef33c167e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:58.507467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c11ad-e59f-49fb-a1b2-07ef33c167e9 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-fe9c11ad-e59f-49fb-a1b2-07ef33c167e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:58.521014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:58.522093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:58.523587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 484/972] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:09:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:58.527996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebe970-2fc4-4966-98d3-3e90c2e498c2 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:58.549348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:58.635369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebe970-2fc4-4966-98d3-3e90c2e498c2 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:58.670332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ee9a3-b59c-40ea-a356-a6d1361df60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:58.671292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ee9a3-b59c-40ea-a356-a6d1361df60e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:58.672310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ee9a3-b59c-40ea-a356-a6d1361df60e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:58.672782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ee9a3-b59c-40ea-a356-a6d1361df60e None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:58.673741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ee9a3-b59c-40ea-a356-a6d1361df60e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:58.674275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ee9a3-b59c-40ea-a356-a6d1361df60e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:58.675918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ee9a3-b59c-40ea-a356-a6d1361df60e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:58.694780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 489/973] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 20:09:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:58.702981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c11ad-e59f-49fb-a1b2-07ef33c167e9 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:58.723991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ee9a3-b59c-40ea-a356-a6d1361df60e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:58.818009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:58.820852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c11ad-e59f-49fb-a1b2-07ef33c167e9 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:58.848615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ee9a3-b59c-40ea-a356-a6d1361df60e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:58.878570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26a29d-97f2-454d-bdf7-7ba4abcd983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:58.879157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a29d-97f2-454d-bdf7-7ba4abcd983f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:58.879441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a29d-97f2-454d-bdf7-7ba4abcd983f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:58.879714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a29d-97f2-454d-bdf7-7ba4abcd983f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:58.889593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:58.890218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:58.890530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:58.890802 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:58.891442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:58.892868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:58.895480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WARNING keystone.server.flask.common [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 05 20:09:58.909267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.sql.core [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 05 20:09:58.909267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 05 20:09:58.909267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [parameters: {'user_id': '4c41937f262248df8e2d0c4f2bb644f7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 05 20:09:58.909267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81337) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 05 20:09:58.909669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application conn.query(q) Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application result.read() Oct 05 20:09:58.930595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 20:09:58.931566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application next(self.gen) Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application yield resource Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application next(self.gen) Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self.session.rollback() Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self.force_reraise() Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise self.value Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application session.commit() Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 05 20:09:58.932509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._prepare_impl() Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self.session.flush() Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._flush(objects) Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application compat.raise_( Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application flush_context.execute() Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rec.execute(self) Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 05 20:09:58.933422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 20:09:58.935043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application conn.query(q) Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application result.read() Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 20:09:58.935870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [parameters: {'user_id': '4c41937f262248df8e2d0c4f2bb644f7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:58.936681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 05 20:09:58.939642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 05 20:09:58.941750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 05 20:09:58.941750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 05 20:09:58.941750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 05 20:09:58.941750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 05 20:09:58.941750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 05 20:09:58.941750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:58.941750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 490/974] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 20:09:58 2022] POST /identity/v3/users => generated 160 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:58.941750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:58.975366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26a29d-97f2-454d-bdf7-7ba4abcd983f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:58.977321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26a29d-97f2-454d-bdf7-7ba4abcd983f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:59.057374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b0c1c-7754-4603-9c58-01fec9e68cbd None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:59.094170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 485/975] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:09:58 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:09:59.098720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26a29d-97f2-454d-bdf7-7ba4abcd983f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:59.107621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390067f-b159-42c7-8757-ba9ae7be5de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:59.109122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390067f-b159-42c7-8757-ba9ae7be5de6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:59.110277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390067f-b159-42c7-8757-ba9ae7be5de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:59.110750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390067f-b159-42c7-8757-ba9ae7be5de6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:59.121166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390067f-b159-42c7-8757-ba9ae7be5de6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:59.123662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390067f-b159-42c7-8757-ba9ae7be5de6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:59.126293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-8390067f-b159-42c7-8757-ba9ae7be5de6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:09:59.139140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:59.140435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 20:09:59.141369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:09:59.143224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 491/976] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:09:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:59.148487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390067f-b159-42c7-8757-ba9ae7be5de6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:59.167935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:59.224913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26a29d-97f2-454d-bdf7-7ba4abcd983f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:59.272609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390067f-b159-42c7-8757-ba9ae7be5de6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:59.273053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98418344-7d5c-4d48-b347-07a26295b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:59.273737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98418344-7d5c-4d48-b347-07a26295b4ad None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:59.274064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98418344-7d5c-4d48-b347-07a26295b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:59.274344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98418344-7d5c-4d48-b347-07a26295b4ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:59.289262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98418344-7d5c-4d48-b347-07a26295b4ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:59.291021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98418344-7d5c-4d48-b347-07a26295b4ad None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:59.310238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b763e-182f-4511-9bb5-2d63e6c06097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-98418344-7d5c-4d48-b347-07a26295b4ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:09:59.310562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:59.311472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:09:59.312926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:09:59.312926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:09:59.312926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:59.312926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 486/977] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:09:59.314559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b763e-182f-4511-9bb5-2d63e6c06097 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:59.314938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b763e-182f-4511-9bb5-2d63e6c06097 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:59.315279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b763e-182f-4511-9bb5-2d63e6c06097 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:59.316070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b763e-182f-4511-9bb5-2d63e6c06097 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:59.316433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b763e-182f-4511-9bb5-2d63e6c06097 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:59.318315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b763e-182f-4511-9bb5-2d63e6c06097 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:59.318635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98418344-7d5c-4d48-b347-07a26295b4ad None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:59.368428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 492/978] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 20:09:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:59.446584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b54b7-e645-4401-a6d8-92edffb01289 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:59.446939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b54b7-e645-4401-a6d8-92edffb01289 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:59.447111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b54b7-e645-4401-a6d8-92edffb01289 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:59.448608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98418344-7d5c-4d48-b347-07a26295b4ad None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:59.448860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 493/979] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:09:59.454579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cd514-e85b-4bf5-b0a1-aa50dd5cb2d4 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:59.454880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cd514-e85b-4bf5-b0a1-aa50dd5cb2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:59.455122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cd514-e85b-4bf5-b0a1-aa50dd5cb2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:59.461015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:59.477713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-5f3cd514-e85b-4bf5-b0a1-aa50dd5cb2d4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:09:59.479851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5f3cd514-e85b-4bf5-b0a1-aa50dd5cb2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:09:59.495081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064140a-b448-4520-ba95-f0124ca3556b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:59.495811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064140a-b448-4520-ba95-f0124ca3556b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:59.496144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064140a-b448-4520-ba95-f0124ca3556b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:59.496420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064140a-b448-4520-ba95-f0124ca3556b None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:59.497499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064140a-b448-4520-ba95-f0124ca3556b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:59.497825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064140a-b448-4520-ba95-f0124ca3556b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:59.500741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064140a-b448-4520-ba95-f0124ca3556b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:59.517149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 487/980] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:09:59.523330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064140a-b448-4520-ba95-f0124ca3556b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:59.544305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:59.569980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 494/981] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:09:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:09:59.648057 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064140a-b448-4520-ba95-f0124ca3556b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:59.685366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa006832-9d78-4c36-9260-90742401f06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:59.686047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa006832-9d78-4c36-9260-90742401f06f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:59.686326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa006832-9d78-4c36-9260-90742401f06f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:59.686598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa006832-9d78-4c36-9260-90742401f06f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:59.694762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa006832-9d78-4c36-9260-90742401f06f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:59.696214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa006832-9d78-4c36-9260-90742401f06f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-aa006832-9d78-4c36-9260-90742401f06f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:09:59.707859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:09:59.708754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:09:59.709734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:09:59.709734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:09:59.709734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:09:59.709734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:09:59.709734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:09:59.709734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:09:59.709734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:09:59.709734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:09:59.709734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 488/982] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:09:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:09:59.715081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cd514-e85b-4bf5-b0a1-aa50dd5cb2d4 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:59.818382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3cd514-e85b-4bf5-b0a1-aa50dd5cb2d4 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:59.846950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa006832-9d78-4c36-9260-90742401f06f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:59.855218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4be48d-1399-4226-8f8e-37a94006fd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:09:59.855835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4be48d-1399-4226-8f8e-37a94006fd05 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:09:59.856113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4be48d-1399-4226-8f8e-37a94006fd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:09:59.856385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4be48d-1399-4226-8f8e-37a94006fd05 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:09:59.857075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4be48d-1399-4226-8f8e-37a94006fd05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:09:59.857376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4be48d-1399-4226-8f8e-37a94006fd05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:09:59.858840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4be48d-1399-4226-8f8e-37a94006fd05 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:09:59.873554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 495/983] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:09:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:09:59.877448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4be48d-1399-4226-8f8e-37a94006fd05 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:09:59.895517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:59.929350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:09:59.976026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa006832-9d78-4c36-9260-90742401f06f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:09:59.987675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4be48d-1399-4226-8f8e-37a94006fd05 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:00.010153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce2c65-ddb4-41c1-944d-d928836097bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:00.010728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce2c65-ddb4-41c1-944d-d928836097bf None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:00.011004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce2c65-ddb4-41c1-944d-d928836097bf None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:00.011275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce2c65-ddb4-41c1-944d-d928836097bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:00.011858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce2c65-ddb4-41c1-944d-d928836097bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:00.013218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce2c65-ddb4-41c1-944d-d928836097bf None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-bdce2c65-ddb4-41c1-944d-d928836097bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:00.022316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:00.023550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:00.024385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 489/984] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:09:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:10:00.027190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b1231-04f8-498f-aafe-0f3b522cdf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:00.027797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b1231-04f8-498f-aafe-0f3b522cdf90 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:00.028310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b1231-04f8-498f-aafe-0f3b522cdf90 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:00.028310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b1231-04f8-498f-aafe-0f3b522cdf90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:00.029029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b1231-04f8-498f-aafe-0f3b522cdf90 None admin] RBAC: Adding query filter params (role.id=8d3762a5bc6a44cfb928f7da69fa6388, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=2314d9416eea4856aec5ac2a0cdfdfc1) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:00.029329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b1231-04f8-498f-aafe-0f3b522cdf90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:00.029431 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce2c65-ddb4-41c1-944d-d928836097bf None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:00.030765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b1231-04f8-498f-aafe-0f3b522cdf90 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:00.038065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 496/985] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:09:59 2022] GET /identity/v3/role_assignments?user.id=2314d9416eea4856aec5ac2a0cdfdfc1&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=8d3762a5bc6a44cfb928f7da69fa6388 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:00.134390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:00.144874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce2c65-ddb4-41c1-944d-d928836097bf None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:00.181026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79534a5-0002-4fd2-97ad-13f07765e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:00.181681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79534a5-0002-4fd2-97ad-13f07765e116 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:00.181961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79534a5-0002-4fd2-97ad-13f07765e116 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:00.182323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79534a5-0002-4fd2-97ad-13f07765e116 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:00.182877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79534a5-0002-4fd2-97ad-13f07765e116 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:00.183182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79534a5-0002-4fd2-97ad-13f07765e116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:00.184580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79534a5-0002-4fd2-97ad-13f07765e116 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:00.195790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 490/986] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:10:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:00.197335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b1231-04f8-498f-aafe-0f3b522cdf90 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:00.302840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b1231-04f8-498f-aafe-0f3b522cdf90 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:00.314939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:00.339665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233da79e-3a37-4609-a884-30b3154c0329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:00.340346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233da79e-3a37-4609-a884-30b3154c0329 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:00.340621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233da79e-3a37-4609-a884-30b3154c0329 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:00.340937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233da79e-3a37-4609-a884-30b3154c0329 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:00.341578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233da79e-3a37-4609-a884-30b3154c0329 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:00.341877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233da79e-3a37-4609-a884-30b3154c0329 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:00.343221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233da79e-3a37-4609-a884-30b3154c0329 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:00.349480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 497/987] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:10:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:00.355450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79534a5-0002-4fd2-97ad-13f07765e116 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:00.459777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79534a5-0002-4fd2-97ad-13f07765e116 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:00.493356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8675f-e711-4fdc-b6aa-6b73bd0ff11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:00.494005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8675f-e711-4fdc-b6aa-6b73bd0ff11a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:00.494277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8675f-e711-4fdc-b6aa-6b73bd0ff11a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:00.494544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8675f-e711-4fdc-b6aa-6b73bd0ff11a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:00.495242 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8675f-e711-4fdc-b6aa-6b73bd0ff11a None admin] RBAC: Adding query filter params (interface=admin, service_id=48fa3f932a924eda82c18bf548589795, region_id=RegionOne) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:00.495606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8675f-e711-4fdc-b6aa-6b73bd0ff11a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:00.497019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8675f-e711-4fdc-b6aa-6b73bd0ff11a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:00.507397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 491/988] 10.222.0.23 () {56 vars in 1249 bytes} [Wed Oct 5 20:10:00 2022] GET /identity/v3/endpoints?service_id=48fa3f932a924eda82c18bf548589795&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:00.508662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233da79e-3a37-4609-a884-30b3154c0329 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:00.617728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233da79e-3a37-4609-a884-30b3154c0329 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:00.651808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98400f-abab-463a-8bf0-33724633644b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:00.652419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98400f-abab-463a-8bf0-33724633644b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:00.652696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98400f-abab-463a-8bf0-33724633644b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:00.652964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98400f-abab-463a-8bf0-33724633644b None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:00.653600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98400f-abab-463a-8bf0-33724633644b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:00.654958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98400f-abab-463a-8bf0-33724633644b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:00.657478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06563b-85c4-4386-b043-ab0d3c18b965 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:00.657808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06563b-85c4-4386-b043-ab0d3c18b965 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:00.658081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06563b-85c4-4386-b043-ab0d3c18b965 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:00.659542 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 492/989] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:00.667514 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 498/990] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 20:10:00 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:10:00.667697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bccd1-a0e2-4b65-8c42-86c43ce159a9 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:00.667697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bccd1-a0e2-4b65-8c42-86c43ce159a9 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:00.667697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bccd1-a0e2-4b65-8c42-86c43ce159a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:00.684649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-992bccd1-a0e2-4b65-8c42-86c43ce159a9 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:00.686171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-992bccd1-a0e2-4b65-8c42-86c43ce159a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:00.704526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:00.769639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 493/991] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:10:01.037919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98400f-abab-463a-8bf0-33724633644b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:01.078631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:01.138251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98400f-abab-463a-8bf0-33724633644b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:01.172832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f15c6e-17cb-416b-939d-bea218941d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:01.173457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f15c6e-17cb-416b-939d-bea218941d67 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:01.173765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f15c6e-17cb-416b-939d-bea218941d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:01.174031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f15c6e-17cb-416b-939d-bea218941d67 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:01.174592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f15c6e-17cb-416b-939d-bea218941d67 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:01.175941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f15c6e-17cb-416b-939d-bea218941d67 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-90f15c6e-17cb-416b-939d-bea218941d67 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:01.184487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:01.185461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 20:10:01.186604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:01.189831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 499/992] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:10:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:01.192317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bccd1-a0e2-4b65-8c42-86c43ce159a9 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:01.292586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bccd1-a0e2-4b65-8c42-86c43ce159a9 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:01.328508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b1c0c-e2a4-4718-84c1-09b93bec6e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:01.329110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1c0c-e2a4-4718-84c1-09b93bec6e0b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:01.329384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1c0c-e2a4-4718-84c1-09b93bec6e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:01.329686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b1c0c-e2a4-4718-84c1-09b93bec6e0b None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:01.330305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b1c0c-e2a4-4718-84c1-09b93bec6e0b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:01.330599 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b1c0c-e2a4-4718-84c1-09b93bec6e0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:01.331954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b1c0c-e2a4-4718-84c1-09b93bec6e0b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:01.341500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 494/993] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:10:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:01.342481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f15c6e-17cb-416b-939d-bea218941d67 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:01.440912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f15c6e-17cb-416b-939d-bea218941d67 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:01.442456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:01.475658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6dbb8-b183-4eac-a862-4f12edac95f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:01.476253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6dbb8-b183-4eac-a862-4f12edac95f6 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:01.476524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6dbb8-b183-4eac-a862-4f12edac95f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:01.476785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6dbb8-b183-4eac-a862-4f12edac95f6 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:01.477395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6dbb8-b183-4eac-a862-4f12edac95f6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:01.477719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6dbb8-b183-4eac-a862-4f12edac95f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:01.479055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6dbb8-b183-4eac-a862-4f12edac95f6 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:01.483911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 500/994] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:10:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:01.489764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b1c0c-e2a4-4718-84c1-09b93bec6e0b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:01.505857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:01.596575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b1c0c-e2a4-4718-84c1-09b93bec6e0b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:01.629902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27922aa-e3fe-4054-aa66-c2f3c76cd4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:01.630498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27922aa-e3fe-4054-aa66-c2f3c76cd4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:01.630772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27922aa-e3fe-4054-aa66-c2f3c76cd4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:01.631041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27922aa-e3fe-4054-aa66-c2f3c76cd4f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:01.631725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27922aa-e3fe-4054-aa66-c2f3c76cd4f4 None admin] RBAC: Adding query filter params (interface=public, service_id=16c3366f827441da92d65aa3fea09d48, region_id=RegionOne) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:01.632049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27922aa-e3fe-4054-aa66-c2f3c76cd4f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:01.633401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27922aa-e3fe-4054-aa66-c2f3c76cd4f4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:01.638538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 495/995] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 20:10:01 2022] GET /identity/v3/endpoints?service_id=16c3366f827441da92d65aa3fea09d48&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:01.644205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6dbb8-b183-4eac-a862-4f12edac95f6 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:01.742217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6dbb8-b183-4eac-a862-4f12edac95f6 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:01.775087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0da6e9-1443-49bc-8640-0e96b95dba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:01.775690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0da6e9-1443-49bc-8640-0e96b95dba3b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:01.775963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0da6e9-1443-49bc-8640-0e96b95dba3b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:01.776232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0da6e9-1443-49bc-8640-0e96b95dba3b None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:01.776823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0da6e9-1443-49bc-8640-0e96b95dba3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:01.778321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0da6e9-1443-49bc-8640-0e96b95dba3b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:01.784610 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 501/996] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 20:10:01 2022] GET /identity/v3/services => generated 1727 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:10:02.238023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d9856-287b-403d-bb95-b6be260f0935 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:02.238530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d9856-287b-403d-bb95-b6be260f0935 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:02.238530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d9856-287b-403d-bb95-b6be260f0935 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:02.239945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 496/997] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:02.244892 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69ec73-b273-4219-b154-8fb9f5812bf0 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:02.245274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69ec73-b273-4219-b154-8fb9f5812bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:02.245471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69ec73-b273-4219-b154-8fb9f5812bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:02.264056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7e69ec73-b273-4219-b154-8fb9f5812bf0 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:02.265365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7e69ec73-b273-4219-b154-8fb9f5812bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:02.289696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:02.337562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 502/998] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:02.603428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d9856-287b-403d-bb95-b6be260f0935 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:02.687712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:02.712386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d9856-287b-403d-bb95-b6be260f0935 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:02.750970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872515e-a0ec-4096-b979-1a3eddafa6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:02.751601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872515e-a0ec-4096-b979-1a3eddafa6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:02.751877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872515e-a0ec-4096-b979-1a3eddafa6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:02.752149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872515e-a0ec-4096-b979-1a3eddafa6d7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:02.752728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872515e-a0ec-4096-b979-1a3eddafa6d7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:02.754132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872515e-a0ec-4096-b979-1a3eddafa6d7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-7872515e-a0ec-4096-b979-1a3eddafa6d7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:02.762984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:02.763950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 20:10:02.764769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:02.768990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 497/999] 10.222.0.23 () {56 vars in 1105 bytes} [Wed Oct 5 20:10:02 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:02.770235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69ec73-b273-4219-b154-8fb9f5812bf0 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:02.832994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb0059-3074-4c16-b9cd-c5ed22ae9477 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:02.833288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb0059-3074-4c16-b9cd-c5ed22ae9477 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:02.833580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb0059-3074-4c16-b9cd-c5ed22ae9477 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:02.835004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 498/1000] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:10:02.840105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8aed2-ce70-40f8-881a-10f68204c077 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:02.840391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8aed2-ce70-40f8-881a-10f68204c077 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:02.840662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8aed2-ce70-40f8-881a-10f68204c077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:02.859769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-55d8aed2-ce70-40f8-881a-10f68204c077 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:02.860878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-55d8aed2-ce70-40f8-881a-10f68204c077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:02.885058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69ec73-b273-4219-b154-8fb9f5812bf0 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:02.920849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186556a5-2d7b-43f7-973e-f5403a471fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:02.921459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186556a5-2d7b-43f7-973e-f5403a471fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:02.922299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186556a5-2d7b-43f7-973e-f5403a471fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:02.922569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186556a5-2d7b-43f7-973e-f5403a471fa1 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:02.923248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186556a5-2d7b-43f7-973e-f5403a471fa1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:02.923552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186556a5-2d7b-43f7-973e-f5403a471fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:02.924904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186556a5-2d7b-43f7-973e-f5403a471fa1 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:02.934322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 503/1001] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:02.935546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186556a5-2d7b-43f7-973e-f5403a471fa1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:02.936879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 499/1002] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:02.962159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:03.041410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186556a5-2d7b-43f7-973e-f5403a471fa1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:03.074849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dcdbd-cd64-4b9b-8770-08efec58db29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:03.075454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dcdbd-cd64-4b9b-8770-08efec58db29 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:03.075726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dcdbd-cd64-4b9b-8770-08efec58db29 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:03.075992 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dcdbd-cd64-4b9b-8770-08efec58db29 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:03.076605 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dcdbd-cd64-4b9b-8770-08efec58db29 None admin] RBAC: Adding query filter params (type=image) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:03.076901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dcdbd-cd64-4b9b-8770-08efec58db29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:03.078811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dcdbd-cd64-4b9b-8770-08efec58db29 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:03.083406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 504/1003] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:03.209043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8aed2-ce70-40f8-881a-10f68204c077 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:03.286878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:03.318549 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8aed2-ce70-40f8-881a-10f68204c077 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:03.352537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fc02c-e39e-4e86-9a95-b7e74ad72d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:03.353178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fc02c-e39e-4e86-9a95-b7e74ad72d60 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:03.353456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fc02c-e39e-4e86-9a95-b7e74ad72d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:03.353783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fc02c-e39e-4e86-9a95-b7e74ad72d60 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:03.354366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fc02c-e39e-4e86-9a95-b7e74ad72d60 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:03.355729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fc02c-e39e-4e86-9a95-b7e74ad72d60 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-4d4fc02c-e39e-4e86-9a95-b7e74ad72d60 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:03.365128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:03.366388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:03.370528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 500/1004] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 20:10:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:03.372589 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dcdbd-cd64-4b9b-8770-08efec58db29 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:03.461482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:03.474940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dcdbd-cd64-4b9b-8770-08efec58db29 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:03.508264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc67088-7a0f-49ef-8b72-d4304b2a1356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:03.508871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc67088-7a0f-49ef-8b72-d4304b2a1356 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:03.509139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc67088-7a0f-49ef-8b72-d4304b2a1356 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:03.509400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc67088-7a0f-49ef-8b72-d4304b2a1356 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:03.510042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc67088-7a0f-49ef-8b72-d4304b2a1356 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:03.510336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc67088-7a0f-49ef-8b72-d4304b2a1356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:03.511671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc67088-7a0f-49ef-8b72-d4304b2a1356 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:03.516585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 505/1005] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:10:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:03.522490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fc02c-e39e-4e86-9a95-b7e74ad72d60 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:03.625074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fc02c-e39e-4e86-9a95-b7e74ad72d60 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:03.657920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8802b-093d-46cc-b435-035cea0d36ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:03.658522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8802b-093d-46cc-b435-035cea0d36ad None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:03.658794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8802b-093d-46cc-b435-035cea0d36ad None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:03.659059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8802b-093d-46cc-b435-035cea0d36ad None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:03.659671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8802b-093d-46cc-b435-035cea0d36ad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:03.659966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8802b-093d-46cc-b435-035cea0d36ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:03.661308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8802b-093d-46cc-b435-035cea0d36ad None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:03.663335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:03.667205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 501/1006] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 20:10:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:03.673386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc67088-7a0f-49ef-8b72-d4304b2a1356 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:03.771700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc67088-7a0f-49ef-8b72-d4304b2a1356 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:03.805399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6c9cd-5de5-4644-828b-6b55c5a77bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:03.806029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6c9cd-5de5-4644-828b-6b55c5a77bce None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:03.806357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6c9cd-5de5-4644-828b-6b55c5a77bce None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:03.806628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6c9cd-5de5-4644-828b-6b55c5a77bce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:03.807291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6c9cd-5de5-4644-828b-6b55c5a77bce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:03.808633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6c9cd-5de5-4644-828b-6b55c5a77bce None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:03.833548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 506/1007] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:10:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:10:03.958812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b855b73-5b26-4c23-bf84-e8c5c2a09656 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:03.959178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b855b73-5b26-4c23-bf84-e8c5c2a09656 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:03.959377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b855b73-5b26-4c23-bf84-e8c5c2a09656 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:03.961056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 502/1008] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:03.967034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7de81-2865-4112-bd81-bb00406aa9c2 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:03.967341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7de81-2865-4112-bd81-bb00406aa9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:03.967596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7de81-2865-4112-bd81-bb00406aa9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:03.987309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f9f7de81-2865-4112-bd81-bb00406aa9c2 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:03.988536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f9f7de81-2865-4112-bd81-bb00406aa9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:03.997105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:04.068924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 507/1009] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:04.335266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b855b73-5b26-4c23-bf84-e8c5c2a09656 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:04.439206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b855b73-5b26-4c23-bf84-e8c5c2a09656 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:04.473986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6cf7d-61c8-42a3-bbfe-7efdd1ac5c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:04.474664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6cf7d-61c8-42a3-bbfe-7efdd1ac5c99 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:04.474943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6cf7d-61c8-42a3-bbfe-7efdd1ac5c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:04.475213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6cf7d-61c8-42a3-bbfe-7efdd1ac5c99 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:04.475839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6cf7d-61c8-42a3-bbfe-7efdd1ac5c99 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:04.477209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6cf7d-61c8-42a3-bbfe-7efdd1ac5c99 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-aee6cf7d-61c8-42a3-bbfe-7efdd1ac5c99 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:04.486702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:04.487639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 20:10:04.488460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:04.492780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 503/1010] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:10:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:04.494040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7de81-2865-4112-bd81-bb00406aa9c2 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:04.595261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7de81-2865-4112-bd81-bb00406aa9c2 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:04.628317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ae999-4c6d-45f3-94d0-9827e0b3d62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:04.628927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ae999-4c6d-45f3-94d0-9827e0b3d62c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:04.629200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ae999-4c6d-45f3-94d0-9827e0b3d62c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:04.629466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ae999-4c6d-45f3-94d0-9827e0b3d62c None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:04.630128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ae999-4c6d-45f3-94d0-9827e0b3d62c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:04.630425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ae999-4c6d-45f3-94d0-9827e0b3d62c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:04.631778 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ae999-4c6d-45f3-94d0-9827e0b3d62c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:04.641671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 508/1011] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:10:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:04.642958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6cf7d-61c8-42a3-bbfe-7efdd1ac5c99 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:04.706732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:04.748897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6cf7d-61c8-42a3-bbfe-7efdd1ac5c99 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:04.783592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88822314-74fc-4fc1-a966-561cd79cd8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:04.784200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88822314-74fc-4fc1-a966-561cd79cd8aa None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:04.784479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88822314-74fc-4fc1-a966-561cd79cd8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:04.784750 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88822314-74fc-4fc1-a966-561cd79cd8aa None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:04.785370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88822314-74fc-4fc1-a966-561cd79cd8aa None admin] RBAC: Adding query filter params (type=compute) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:04.785703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88822314-74fc-4fc1-a966-561cd79cd8aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:04.787072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88822314-74fc-4fc1-a966-561cd79cd8aa None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:04.797263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 504/1012] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 20:10:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:04.798252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ae999-4c6d-45f3-94d0-9827e0b3d62c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:04.801083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:04.905115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ae999-4c6d-45f3-94d0-9827e0b3d62c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:04.941141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a7990-4e24-4057-bb30-d3fc87d30114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:04.941780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a7990-4e24-4057-bb30-d3fc87d30114 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:04.942060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a7990-4e24-4057-bb30-d3fc87d30114 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:04.942336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a7990-4e24-4057-bb30-d3fc87d30114 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:04.942990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a7990-4e24-4057-bb30-d3fc87d30114 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:04.944339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a7990-4e24-4057-bb30-d3fc87d30114 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:04.973549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 509/1013] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:10:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:10:05.252690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57855d-4e08-4181-9647-46db6f1fcd03 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:05.252992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57855d-4e08-4181-9647-46db6f1fcd03 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:05.253166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57855d-4e08-4181-9647-46db6f1fcd03 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:05.254653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 505/1014] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:05.259686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270ee92-6cac-4838-b4ff-a0a0986c288f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:05.259975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270ee92-6cac-4838-b4ff-a0a0986c288f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:05.260325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270ee92-6cac-4838-b4ff-a0a0986c288f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:05.278625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-3270ee92-6cac-4838-b4ff-a0a0986c288f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:05.279785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3270ee92-6cac-4838-b4ff-a0a0986c288f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:05.333497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:05.356539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 510/1015] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:05 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:05.621526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57855d-4e08-4181-9647-46db6f1fcd03 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:05.723766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57855d-4e08-4181-9647-46db6f1fcd03 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:05.750445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:05.759377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17daf370-2622-4abd-8de6-98c91e019c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:05.759981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daf370-2622-4abd-8de6-98c91e019c71 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:05.760257 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daf370-2622-4abd-8de6-98c91e019c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:05.760519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daf370-2622-4abd-8de6-98c91e019c71 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:05.761171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daf370-2622-4abd-8de6-98c91e019c71 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:05.762553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daf370-2622-4abd-8de6-98c91e019c71 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:05.786093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 506/1016] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:10:05 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:10:05.999253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0824503-3c32-4aeb-a9f6-5bc68ada9d69 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:05.999781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0824503-3c32-4aeb-a9f6-5bc68ada9d69 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:05.999781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0824503-3c32-4aeb-a9f6-5bc68ada9d69 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:06.001126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 511/1017] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:10:06.006672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c672c48-704d-4023-a2cd-0a34a71f5c7d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:06.006963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c672c48-704d-4023-a2cd-0a34a71f5c7d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:06.007240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c672c48-704d-4023-a2cd-0a34a71f5c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:06.025754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8c672c48-704d-4023-a2cd-0a34a71f5c7d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:06.026938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8c672c48-704d-4023-a2cd-0a34a71f5c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:06.103155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 507/1018] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:06.374376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0824503-3c32-4aeb-a9f6-5bc68ada9d69 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:06.477396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0824503-3c32-4aeb-a9f6-5bc68ada9d69 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:06.511573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018657a7-0832-466d-85f5-73246e2a2924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:06.512205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018657a7-0832-466d-85f5-73246e2a2924 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:06.512534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018657a7-0832-466d-85f5-73246e2a2924 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:06.512808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018657a7-0832-466d-85f5-73246e2a2924 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:06.513937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018657a7-0832-466d-85f5-73246e2a2924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:06.515956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018657a7-0832-466d-85f5-73246e2a2924 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:06.517704 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 512/1019] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:06.523382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c672c48-704d-4023-a2cd-0a34a71f5c7d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:06.625300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c672c48-704d-4023-a2cd-0a34a71f5c7d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:06.639308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:06.663061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9d910-1fa9-43ea-a1ce-a63fd57aa221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:06.663755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9d910-1fa9-43ea-a1ce-a63fd57aa221 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:06.664044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9d910-1fa9-43ea-a1ce-a63fd57aa221 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:06.664309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9d910-1fa9-43ea-a1ce-a63fd57aa221 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:06.664969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9d910-1fa9-43ea-a1ce-a63fd57aa221 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:06.666487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9d910-1fa9-43ea-a1ce-a63fd57aa221 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:06.696930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 508/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:10:06 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:10:07.223639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95912e6d-0dc2-4c9e-8e0e-8bf5a58bdb84 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:07.224151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95912e6d-0dc2-4c9e-8e0e-8bf5a58bdb84 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:07.224151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95912e6d-0dc2-4c9e-8e0e-8bf5a58bdb84 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:07.225585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 513/1021] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:07.230670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e7a73-88ef-4cae-bc0f-01c095cd8319 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:07.231042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e7a73-88ef-4cae-bc0f-01c095cd8319 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:07.231389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e7a73-88ef-4cae-bc0f-01c095cd8319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:07.250039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7d7e7a73-88ef-4cae-bc0f-01c095cd8319 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:07.251271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7d7e7a73-88ef-4cae-bc0f-01c095cd8319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:07.318503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 509/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:07.593455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95912e6d-0dc2-4c9e-8e0e-8bf5a58bdb84 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:07.650435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:07.699891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95912e6d-0dc2-4c9e-8e0e-8bf5a58bdb84 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:07.732528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f7ae2-481f-438b-933a-5cbbd9a6f61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:07.733151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f7ae2-481f-438b-933a-5cbbd9a6f61a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:07.733423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f7ae2-481f-438b-933a-5cbbd9a6f61a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:07.733714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f7ae2-481f-438b-933a-5cbbd9a6f61a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:07.744202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f7ae2-481f-438b-933a-5cbbd9a6f61a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-527f7ae2-481f-438b-933a-5cbbd9a6f61a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:10:07.748907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:07.750379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:07.751231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 514/1023] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 20:10:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:07.757919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e7a73-88ef-4cae-bc0f-01c095cd8319 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:07.856468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:07.869026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e7a73-88ef-4cae-bc0f-01c095cd8319 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:07.907369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd557b-0fd0-4e79-8b0f-67e9bb80dea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:07.907975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd557b-0fd0-4e79-8b0f-67e9bb80dea3 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:07.908256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd557b-0fd0-4e79-8b0f-67e9bb80dea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:07.908521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd557b-0fd0-4e79-8b0f-67e9bb80dea3 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:07.909174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd557b-0fd0-4e79-8b0f-67e9bb80dea3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:07.909471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd557b-0fd0-4e79-8b0f-67e9bb80dea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:07.910854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd557b-0fd0-4e79-8b0f-67e9bb80dea3 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:07.923660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 510/1024] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 20:10:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:07.924949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f7ae2-481f-438b-933a-5cbbd9a6f61a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:07.943471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011237-b563-4456-9a05-56d4c9806224 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:07.944020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011237-b563-4456-9a05-56d4c9806224 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:07.944354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33011237-b563-4456-9a05-56d4c9806224 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:07.946468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 511/1025] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:10:07.951274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3238eb2-ebe6-477e-89fb-3207ee27e52c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:07.951553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3238eb2-ebe6-477e-89fb-3207ee27e52c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:07.951827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3238eb2-ebe6-477e-89fb-3207ee27e52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:07.975548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f3238eb2-ebe6-477e-89fb-3207ee27e52c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:07.977039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f3238eb2-ebe6-477e-89fb-3207ee27e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:08.032324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f7ae2-481f-438b-933a-5cbbd9a6f61a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:08.048262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:08.051153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 512/1026] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:10:08.071723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd5565-b697-426b-b405-3d9b96b790a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:08.072354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd5565-b697-426b-b405-3d9b96b790a8 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.072632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd5565-b697-426b-b405-3d9b96b790a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.072920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd5565-b697-426b-b405-3d9b96b790a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.083382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd5565-b697-426b-b405-3d9b96b790a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:08.084882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd5565-b697-426b-b405-3d9b96b790a8 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-fadd5565-b697-426b-b405-3d9b96b790a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:10:08.100206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:08.101121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:10:08.102054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:10:08.102054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:08.102054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:08.102054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 515/1027] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:08.107006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3238eb2-ebe6-477e-89fb-3207ee27e52c None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.196521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:08.210185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3238eb2-ebe6-477e-89fb-3207ee27e52c None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:08.244627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a158b9-96e4-4ba9-97a3-da2600bedb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:08.245232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a158b9-96e4-4ba9-97a3-da2600bedb9d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.245525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a158b9-96e4-4ba9-97a3-da2600bedb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.245809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a158b9-96e4-4ba9-97a3-da2600bedb9d None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.246444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a158b9-96e4-4ba9-97a3-da2600bedb9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:08.246746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a158b9-96e4-4ba9-97a3-da2600bedb9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:08.248110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a158b9-96e4-4ba9-97a3-da2600bedb9d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:08.264147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 513/1028] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:08.266162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd5565-b697-426b-b405-3d9b96b790a8 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.316539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a158b9-96e4-4ba9-97a3-da2600bedb9d None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.375280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd5565-b697-426b-b405-3d9b96b790a8 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:08.410693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b185d-ea2a-4bbc-857b-5372b0668d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:08.411312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b185d-ea2a-4bbc-857b-5372b0668d0f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.411588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b185d-ea2a-4bbc-857b-5372b0668d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.411853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b185d-ea2a-4bbc-857b-5372b0668d0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.419236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b185d-ea2a-4bbc-857b-5372b0668d0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:08.420359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a158b9-96e4-4ba9-97a3-da2600bedb9d None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:08.420657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b185d-ea2a-4bbc-857b-5372b0668d0f None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-e65b185d-ea2a-4bbc-857b-5372b0668d0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:10:08.432600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:08.439102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 516/1029] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:08.440533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b185d-ea2a-4bbc-857b-5372b0668d0f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.466797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:08.469266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8abc7-be56-4f62-946f-723d75220235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:08.470741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8abc7-be56-4f62-946f-723d75220235 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.471397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8abc7-be56-4f62-946f-723d75220235 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.471734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8abc7-be56-4f62-946f-723d75220235 None admin] PATH_INFO: `/v3/services/image` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.472321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8abc7-be56-4f62-946f-723d75220235 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:08.473704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8abc7-be56-4f62-946f-723d75220235 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-ead8abc7-be56-4f62-946f-723d75220235 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:08.484425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:08.485456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:08.491189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 514/1030] 10.222.0.23 () {56 vars in 1105 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:10:08.493320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8abc7-be56-4f62-946f-723d75220235 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.509507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:08.550564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b185d-ea2a-4bbc-857b-5372b0668d0f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:08.589385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11640b-a53f-464c-966a-0f0a1fb0f806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:08.590030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11640b-a53f-464c-966a-0f0a1fb0f806 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.590452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11640b-a53f-464c-966a-0f0a1fb0f806 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.590729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11640b-a53f-464c-966a-0f0a1fb0f806 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.591470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11640b-a53f-464c-966a-0f0a1fb0f806 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:08.591775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11640b-a53f-464c-966a-0f0a1fb0f806 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:08.593314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11640b-a53f-464c-966a-0f0a1fb0f806 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:08.601368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8abc7-be56-4f62-946f-723d75220235 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:08.613114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 517/1031] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:08.618452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11640b-a53f-464c-966a-0f0a1fb0f806 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.640233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef15c46-ab19-41a0-986c-7b8284922a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:08.641065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef15c46-ab19-41a0-986c-7b8284922a4c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.641349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef15c46-ab19-41a0-986c-7b8284922a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.641651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef15c46-ab19-41a0-986c-7b8284922a4c None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.642452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef15c46-ab19-41a0-986c-7b8284922a4c None admin] RBAC: Adding query filter params (name=image) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:08.643109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef15c46-ab19-41a0-986c-7b8284922a4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:08.644510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef15c46-ab19-41a0-986c-7b8284922a4c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:08.654831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 515/1032] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:08.656343 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef15c46-ab19-41a0-986c-7b8284922a4c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.707348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:08.729134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11640b-a53f-464c-966a-0f0a1fb0f806 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:08.738016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:08.763001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef15c46-ab19-41a0-986c-7b8284922a4c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:08.765686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b476ba-7e0a-484a-82d8-33a41d982673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:08.766386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b476ba-7e0a-484a-82d8-33a41d982673 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.766836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b476ba-7e0a-484a-82d8-33a41d982673 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.767215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b476ba-7e0a-484a-82d8-33a41d982673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.798678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba1d44-e0f1-4ff1-a880-2e22a7b2cb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:08.799310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba1d44-e0f1-4ff1-a880-2e22a7b2cb4b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.799595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba1d44-e0f1-4ff1-a880-2e22a7b2cb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.799868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba1d44-e0f1-4ff1-a880-2e22a7b2cb4b None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.800525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba1d44-e0f1-4ff1-a880-2e22a7b2cb4b None admin] RBAC: Adding query filter params (type=image) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:08.800832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba1d44-e0f1-4ff1-a880-2e22a7b2cb4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:08.802279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ba1d44-e0f1-4ff1-a880-2e22a7b2cb4b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:08.812792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 516/1033] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:08.814903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba1d44-e0f1-4ff1-a880-2e22a7b2cb4b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.848232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b476ba-7e0a-484a-82d8-33a41d982673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:08.849797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b476ba-7e0a-484a-82d8-33a41d982673 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:08.922977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba1d44-e0f1-4ff1-a880-2e22a7b2cb4b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:08.946796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:08.955866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f236d10-307a-4fca-b9fa-4fc7a6843e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:08.956465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f236d10-307a-4fca-b9fa-4fc7a6843e30 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.956735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f236d10-307a-4fca-b9fa-4fc7a6843e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.957006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f236d10-307a-4fca-b9fa-4fc7a6843e30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.957727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f236d10-307a-4fca-b9fa-4fc7a6843e30 None admin] RBAC: Adding query filter params (interface=public, service_id=9d29adbfc7454b88b72442533b55feaa, region_id=RegionOne) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:08.958031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f236d10-307a-4fca-b9fa-4fc7a6843e30 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:08.959378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f236d10-307a-4fca-b9fa-4fc7a6843e30 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:08.964210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 517/1034] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/endpoints?service_id=9d29adbfc7454b88b72442533b55feaa&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:08.965607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a92d60-4bb0-406f-94ce-470f04f1000a None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:08.966035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a92d60-4bb0-406f-94ce-470f04f1000a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:08.966302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 518/1035] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 05 20:10:08.966628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a92d60-4bb0-406f-94ce-470f04f1000a None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:08.967955 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 518/1036] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:08.970485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b476ba-7e0a-484a-82d8-33a41d982673 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.973285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a92d60-4bb0-406f-94ce-470f04f1000a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:08.977457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:09.072811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b476ba-7e0a-484a-82d8-33a41d982673 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:09.073735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a92d60-4bb0-406f-94ce-470f04f1000a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:09.106336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8d676-a1d6-4a78-bba5-bb882438f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:09.106663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8a06a-5cc1-43ac-8bae-902364ddfae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:09.106931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8d676-a1d6-4a78-bba5-bb882438f6aa None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:09.107206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8d676-a1d6-4a78-bba5-bb882438f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:09.107299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8a06a-5cc1-43ac-8bae-902364ddfae8 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:09.107477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8d676-a1d6-4a78-bba5-bb882438f6aa None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:09.107575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8a06a-5cc1-43ac-8bae-902364ddfae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:09.107800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8a06a-5cc1-43ac-8bae-902364ddfae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:09.108068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8d676-a1d6-4a78-bba5-bb882438f6aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:09.109468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8d676-a1d6-4a78-bba5-bb882438f6aa None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:09.116090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 519/1037] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:10:09.117065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd818e0-c9dc-401e-a200-10bb47cac90f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:09.117365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd818e0-c9dc-401e-a200-10bb47cac90f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:09.117649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd818e0-c9dc-401e-a200-10bb47cac90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:09.118321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8a06a-5cc1-43ac-8bae-902364ddfae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:09.119771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8a06a-5cc1-43ac-8bae-902364ddfae8 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-83f8a06a-5cc1-43ac-8bae-902364ddfae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:10:09.135834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:09.136857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:10:09.137762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:10:09.137762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:09.137762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:09.137945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ffd818e0-c9dc-401e-a200-10bb47cac90f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:09.138020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 519/1038] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:09.138087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ffd818e0-c9dc-401e-a200-10bb47cac90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:09.143267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8a06a-5cc1-43ac-8bae-902364ddfae8 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:09.189641 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:09.212727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:09.216677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 520/1039] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:09.262494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8a06a-5cc1-43ac-8bae-902364ddfae8 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:09.299470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6cfce-c846-45f0-8e64-46443452af2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:09.300081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6cfce-c846-45f0-8e64-46443452af2f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:09.300362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6cfce-c846-45f0-8e64-46443452af2f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:09.300636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6cfce-c846-45f0-8e64-46443452af2f None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:09.301273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6cfce-c846-45f0-8e64-46443452af2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:09.301604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6cfce-c846-45f0-8e64-46443452af2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:09.302972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6cfce-c846-45f0-8e64-46443452af2f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:09.316717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 520/1040] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:09.322128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd818e0-c9dc-401e-a200-10bb47cac90f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:09.424560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd818e0-c9dc-401e-a200-10bb47cac90f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:09.457461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff384a52-242d-41a3-8b41-044289f755ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:09.458042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff384a52-242d-41a3-8b41-044289f755ae None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:09.458314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff384a52-242d-41a3-8b41-044289f755ae None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:09.458581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff384a52-242d-41a3-8b41-044289f755ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:09.465138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff384a52-242d-41a3-8b41-044289f755ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:09.466584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff384a52-242d-41a3-8b41-044289f755ae None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-ff384a52-242d-41a3-8b41-044289f755ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:10:09.477917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:09.478814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:09.479692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:10:09.479692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:10:09.479692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:10:09.479692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:10:09.479692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:10:09.479692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:10:09.479692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:10:09.479692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:09.479692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 521/1041] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:10:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:09.486163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6cfce-c846-45f0-8e64-46443452af2f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:09.486547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff384a52-242d-41a3-8b41-044289f755ae None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:09.520609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:09.586797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6cfce-c846-45f0-8e64-46443452af2f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:09.590314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff384a52-242d-41a3-8b41-044289f755ae None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:09.603649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:09.624359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf9ab4-e63f-4d97-93ac-ada094969d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:09.625006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf9ab4-e63f-4d97-93ac-ada094969d2f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:09.625280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf9ab4-e63f-4d97-93ac-ada094969d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:09.625569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf9ab4-e63f-4d97-93ac-ada094969d2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:09.626456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c782d-04f3-40f3-9dbd-e716e20cf5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:09.626657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf9ab4-e63f-4d97-93ac-ada094969d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:09.627083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c782d-04f3-40f3-9dbd-e716e20cf5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:09.627356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c782d-04f3-40f3-9dbd-e716e20cf5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:09.627630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c782d-04f3-40f3-9dbd-e716e20cf5b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:09.628044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf9ab4-e63f-4d97-93ac-ada094969d2f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:09.628294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c782d-04f3-40f3-9dbd-e716e20cf5b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:09.628592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c782d-04f3-40f3-9dbd-e716e20cf5b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:09.629793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 521/1042] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:09.633957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c782d-04f3-40f3-9dbd-e716e20cf5b3 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:09.635190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf9ab4-e63f-4d97-93ac-ada094969d2f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:09.651498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 522/1043] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:10:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:09.652479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c782d-04f3-40f3-9dbd-e716e20cf5b3 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:09.747914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf9ab4-e63f-4d97-93ac-ada094969d2f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:09.750684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:09.757318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c782d-04f3-40f3-9dbd-e716e20cf5b3 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:09.791608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120509b1-b0e6-4e5c-9b4e-5dd6c71cb0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:09.792299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120509b1-b0e6-4e5c-9b4e-5dd6c71cb0c8 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:09.792644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120509b1-b0e6-4e5c-9b4e-5dd6c71cb0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:09.792982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120509b1-b0e6-4e5c-9b4e-5dd6c71cb0c8 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:09.793765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120509b1-b0e6-4e5c-9b4e-5dd6c71cb0c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:09.794441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a0fb2-642c-4dfd-bf8c-0aece6dccb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:09.795108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a0fb2-642c-4dfd-bf8c-0aece6dccb67 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:09.795361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120509b1-b0e6-4e5c-9b4e-5dd6c71cb0c8 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:09.795517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a0fb2-642c-4dfd-bf8c-0aece6dccb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:09.795849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a0fb2-642c-4dfd-bf8c-0aece6dccb67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:09.796750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a0fb2-642c-4dfd-bf8c-0aece6dccb67 None admin] RBAC: Adding query filter params (role.id=8d3762a5bc6a44cfb928f7da69fa6388, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=286f4e203e7a4a55bca0a58d061af2bd) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:09.797224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a0fb2-642c-4dfd-bf8c-0aece6dccb67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:09.798753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a0fb2-642c-4dfd-bf8c-0aece6dccb67 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:09.804665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 523/1044] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:10:09 2022] GET /identity/v3/role_assignments?user.id=286f4e203e7a4a55bca0a58d061af2bd&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=8d3762a5bc6a44cfb928f7da69fa6388 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:09.833586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 522/1045] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:10:09 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:10:11.295599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60a6c-98e2-417d-9c52-ca9e602c23d0 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:11.296118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60a6c-98e2-417d-9c52-ca9e602c23d0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:11.296118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60a6c-98e2-417d-9c52-ca9e602c23d0 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:11.298204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 524/1046] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:11.303333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ebce2-4d69-4776-acf5-318075f94005 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:11.303743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ebce2-4d69-4776-acf5-318075f94005 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:11.303910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ebce2-4d69-4776-acf5-318075f94005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:11.322684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ba5ebce2-4d69-4776-acf5-318075f94005 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:11.323863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ba5ebce2-4d69-4776-acf5-318075f94005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:11.359256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:11.394996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 523/1047] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:11.662523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d60a6c-98e2-417d-9c52-ca9e602c23d0 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:11.766261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d60a6c-98e2-417d-9c52-ca9e602c23d0 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:11.800082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3424d-cb86-435d-b38b-a97ab6a5a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:11.800710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3424d-cb86-435d-b38b-a97ab6a5a4ec None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:11.800989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3424d-cb86-435d-b38b-a97ab6a5a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:11.801264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3424d-cb86-435d-b38b-a97ab6a5a4ec None admin] PATH_INFO: `/v3/services/image` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:11.801867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3424d-cb86-435d-b38b-a97ab6a5a4ec None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:11.803221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3424d-cb86-435d-b38b-a97ab6a5a4ec None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-dea3424d-cb86-435d-b38b-a97ab6a5a4ec None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:11.811876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:11.812856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 20:10:11.814093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:11.815415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 525/1048] 10.222.0.23 () {56 vars in 1105 bytes} [Wed Oct 5 20:10:11 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:11.819094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ebce2-4d69-4776-acf5-318075f94005 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:11.931533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ebce2-4d69-4776-acf5-318075f94005 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:11.969354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa94f3-8275-4325-81a0-3bf2e88b0d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:11.969998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa94f3-8275-4325-81a0-3bf2e88b0d4a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:11.970273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa94f3-8275-4325-81a0-3bf2e88b0d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:11.970544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa94f3-8275-4325-81a0-3bf2e88b0d4a None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:11.971171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa94f3-8275-4325-81a0-3bf2e88b0d4a None admin] RBAC: Adding query filter params (name=image) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:11.971470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa94f3-8275-4325-81a0-3bf2e88b0d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:11.972837 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa94f3-8275-4325-81a0-3bf2e88b0d4a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:11.982674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 524/1049] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:11.984697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc6d8b-7bca-4552-9ad9-84de7297ed4f None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:11.984981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc6d8b-7bca-4552-9ad9-84de7297ed4f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:11.985251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc6d8b-7bca-4552-9ad9-84de7297ed4f None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:11.985571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa94f3-8275-4325-81a0-3bf2e88b0d4a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:11.986701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 526/1050] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:10:11.991675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8ff67-9446-4466-b3bc-a2ff0d0fde41 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:11.991960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8ff67-9446-4466-b3bc-a2ff0d0fde41 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:11.992238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8ff67-9446-4466-b3bc-a2ff0d0fde41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.010278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-aca8ff67-9446-4466-b3bc-a2ff0d0fde41 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:12.011463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-aca8ff67-9446-4466-b3bc-a2ff0d0fde41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:12.021321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:12.031891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:12.083628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 527/1051] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:10:12.084884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95575f2-185e-4d79-8758-977427e04efd None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.085169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95575f2-185e-4d79-8758-977427e04efd None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.085437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95575f2-185e-4d79-8758-977427e04efd None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.086867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 528/1052] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:12.092324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7be376-3578-4d33-8785-6c9cdb23c150 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.092324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7be376-3578-4d33-8785-6c9cdb23c150 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.092709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7be376-3578-4d33-8785-6c9cdb23c150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.098843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa94f3-8275-4325-81a0-3bf2e88b0d4a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:12.114735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ca7be376-3578-4d33-8785-6c9cdb23c150 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:12.116138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ca7be376-3578-4d33-8785-6c9cdb23c150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:12.135152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c8573-af77-438f-8c01-0cd697646499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:12.135755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c8573-af77-438f-8c01-0cd697646499 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.136027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c8573-af77-438f-8c01-0cd697646499 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.136291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c8573-af77-438f-8c01-0cd697646499 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.136912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c8573-af77-438f-8c01-0cd697646499 None admin] RBAC: Adding query filter params (type=image) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:12.137208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c8573-af77-438f-8c01-0cd697646499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:12.138593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c8573-af77-438f-8c01-0cd697646499 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:12.144117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 525/1053] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:12.149793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c8573-af77-438f-8c01-0cd697646499 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:12.201612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 529/1054] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:10:12.250675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:12.254610 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c8573-af77-438f-8c01-0cd697646499 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:12.289386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3d964-5224-4e2e-b4e1-a060f3a947b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:12.290008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3d964-5224-4e2e-b4e1-a060f3a947b6 None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.290288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3d964-5224-4e2e-b4e1-a060f3a947b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.290557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3d964-5224-4e2e-b4e1-a060f3a947b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.291212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3d964-5224-4e2e-b4e1-a060f3a947b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:12.292573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3d964-5224-4e2e-b4e1-a060f3a947b6 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:12.313422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 526/1055] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:10:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:10:12.362712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7be376-3578-4d33-8785-6c9cdb23c150 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:12.454516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:12.458820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3d964-5224-4e2e-b4e1-a060f3a947b6 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:12.473999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7be376-3578-4d33-8785-6c9cdb23c150 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:12.524842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12ab2b-92d3-473d-9c17-43c59a7ad439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:12.525611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ab2b-92d3-473d-9c17-43c59a7ad439 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.525780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ab2b-92d3-473d-9c17-43c59a7ad439 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.526042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ab2b-92d3-473d-9c17-43c59a7ad439 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.536736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12ab2b-92d3-473d-9c17-43c59a7ad439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-fb12ab2b-92d3-473d-9c17-43c59a7ad439 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:10:12.541433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:12.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:12.543191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 530/1056] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 20:10:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:12.550534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12ab2b-92d3-473d-9c17-43c59a7ad439 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:12.572420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3d964-5224-4e2e-b4e1-a060f3a947b6 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:12.608877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2811b-6b57-4c72-bf0b-c491b04482e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:12.609504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2811b-6b57-4c72-bf0b-c491b04482e2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.609798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2811b-6b57-4c72-bf0b-c491b04482e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.610064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2811b-6b57-4c72-bf0b-c491b04482e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.621670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2811b-6b57-4c72-bf0b-c491b04482e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-40b2811b-6b57-4c72-bf0b-c491b04482e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:10:12.626913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:12.628009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:12.633734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 527/1057] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:10:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:10:12.635446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2811b-6b57-4c72-bf0b-c491b04482e2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:12.653266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:12.664094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12ab2b-92d3-473d-9c17-43c59a7ad439 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:12.700225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06537c-e9a6-4084-a65b-7f49c96519a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:12.700829 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06537c-e9a6-4084-a65b-7f49c96519a2 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.701102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06537c-e9a6-4084-a65b-7f49c96519a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.701372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06537c-e9a6-4084-a65b-7f49c96519a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.702053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06537c-e9a6-4084-a65b-7f49c96519a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:12.702348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06537c-e9a6-4084-a65b-7f49c96519a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:12.704347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06537c-e9a6-4084-a65b-7f49c96519a2 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:12.717148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 531/1058] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 20:10:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:12.717963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06537c-e9a6-4084-a65b-7f49c96519a2 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:12.720559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:12.745280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2811b-6b57-4c72-bf0b-c491b04482e2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:12.782447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576c96f-ea21-4004-8cb7-4a2ec3e837da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:12.783076 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576c96f-ea21-4004-8cb7-4a2ec3e837da None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.783352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576c96f-ea21-4004-8cb7-4a2ec3e837da None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.783621 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576c96f-ea21-4004-8cb7-4a2ec3e837da None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.784277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576c96f-ea21-4004-8cb7-4a2ec3e837da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:12.784575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576c96f-ea21-4004-8cb7-4a2ec3e837da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:12.786002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576c96f-ea21-4004-8cb7-4a2ec3e837da None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:12.794941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 528/1059] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:10:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:12.801470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576c96f-ea21-4004-8cb7-4a2ec3e837da None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:12.830510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06537c-e9a6-4084-a65b-7f49c96519a2 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:12.866165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d8454-4d5f-4a75-a3b4-92b1b8d483c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:12.866806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d8454-4d5f-4a75-a3b4-92b1b8d483c5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.867085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d8454-4d5f-4a75-a3b4-92b1b8d483c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.867370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d8454-4d5f-4a75-a3b4-92b1b8d483c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.880263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d8454-4d5f-4a75-a3b4-92b1b8d483c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:12.881729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d8454-4d5f-4a75-a3b4-92b1b8d483c5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:12.886047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-420d8454-4d5f-4a75-a3b4-92b1b8d483c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:10:12.899857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:12.900985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:10:12.902038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:10:12.902038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:12.902038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:12.903921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 532/1060] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:10:12.909114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d8454-4d5f-4a75-a3b4-92b1b8d483c5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:12.915025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576c96f-ea21-4004-8cb7-4a2ec3e837da None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:12.951249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78894843-237b-4050-a924-3f40969dda21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:12.952204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78894843-237b-4050-a924-3f40969dda21 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:12.952521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78894843-237b-4050-a924-3f40969dda21 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:12.952794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78894843-237b-4050-a924-3f40969dda21 None admin] PATH_INFO: `/v3/users/9f558f088da54d14b9c2d6a9010ae421` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:12.966748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78894843-237b-4050-a924-3f40969dda21 None admin] RBAC: Authorizing `identity:get_user(user_id=9f558f088da54d14b9c2d6a9010ae421)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:12.967600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:12.968489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78894843-237b-4050-a924-3f40969dda21 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:12.974853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 529/1061] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:12 2022] GET /identity/v3/users/9f558f088da54d14b9c2d6a9010ae421 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:12.975863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78894843-237b-4050-a924-3f40969dda21 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:13.019511 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d8454-4d5f-4a75-a3b4-92b1b8d483c5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:13.055505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb2bae-4023-45e4-bbcf-4175a5fd92bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:13.056117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb2bae-4023-45e4-bbcf-4175a5fd92bf None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:13.056396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb2bae-4023-45e4-bbcf-4175a5fd92bf None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:13.056668 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb2bae-4023-45e4-bbcf-4175a5fd92bf None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:13.057307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb2bae-4023-45e4-bbcf-4175a5fd92bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:13.057690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb2bae-4023-45e4-bbcf-4175a5fd92bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:13.059002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb2bae-4023-45e4-bbcf-4175a5fd92bf None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:13.072316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 533/1062] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:13.077989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb2bae-4023-45e4-bbcf-4175a5fd92bf None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:13.081109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78894843-237b-4050-a924-3f40969dda21 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:13.118660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e96ca-0b7f-43d2-b5bf-da86706eb369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:13.119296 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e96ca-0b7f-43d2-b5bf-da86706eb369 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:13.119581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e96ca-0b7f-43d2-b5bf-da86706eb369 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:13.119847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e96ca-0b7f-43d2-b5bf-da86706eb369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:13.122828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:13.184318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb2bae-4023-45e4-bbcf-4175a5fd92bf None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:13.193550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e96ca-0b7f-43d2-b5bf-da86706eb369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:13.195001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e96ca-0b7f-43d2-b5bf-da86706eb369 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:13.200336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:13.228128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b6f21-3cd3-45e4-889b-66e203eb66fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:13.229132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b6f21-3cd3-45e4-889b-66e203eb66fb None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:13.229585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b6f21-3cd3-45e4-889b-66e203eb66fb None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:13.230001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b6f21-3cd3-45e4-889b-66e203eb66fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:13.239503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b6f21-3cd3-45e4-889b-66e203eb66fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:13.242025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b6f21-3cd3-45e4-889b-66e203eb66fb None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-407b6f21-3cd3-45e4-889b-66e203eb66fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:10:13.263695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:13.264672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:10:13.265673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:13.266622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 534/1063] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 20:10:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:10:13.271075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b6f21-3cd3-45e4-889b-66e203eb66fb None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:13.310770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 530/1064] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 05 20:10:13.316815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e96ca-0b7f-43d2-b5bf-da86706eb369 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:13.350805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:13.379467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b6f21-3cd3-45e4-889b-66e203eb66fb None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:13.414815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c99b0-c71e-4fd0-b332-049ea3c9e61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:13.415544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c99b0-c71e-4fd0-b332-049ea3c9e61b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:13.415847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c99b0-c71e-4fd0-b332-049ea3c9e61b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:13.416283 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c99b0-c71e-4fd0-b332-049ea3c9e61b None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:13.416923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c99b0-c71e-4fd0-b332-049ea3c9e61b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:13.417217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c99b0-c71e-4fd0-b332-049ea3c9e61b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:13.419341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c99b0-c71e-4fd0-b332-049ea3c9e61b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:13.425205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e96ca-0b7f-43d2-b5bf-da86706eb369 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:13.441405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 535/1065] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 20:10:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:13.446904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c99b0-c71e-4fd0-b332-049ea3c9e61b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:13.462056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59e71f-3617-4b0f-a73a-9005dcdc9e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:13.462684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59e71f-3617-4b0f-a73a-9005dcdc9e72 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:13.462960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59e71f-3617-4b0f-a73a-9005dcdc9e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:13.463227 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59e71f-3617-4b0f-a73a-9005dcdc9e72 None admin] PATH_INFO: `/v3/projects/024302e671b94c1ab0275621f3fdb02b` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:13.464312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba59e71f-3617-4b0f-a73a-9005dcdc9e72 None admin] RBAC: Authorizing `identity:get_project(project_id=024302e671b94c1ab0275621f3fdb02b)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:13.465761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba59e71f-3617-4b0f-a73a-9005dcdc9e72 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:13.472746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 531/1066] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:10:13 2022] GET /identity/v3/projects/024302e671b94c1ab0275621f3fdb02b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:13.474022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59e71f-3617-4b0f-a73a-9005dcdc9e72 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:13.483260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:13.557811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c99b0-c71e-4fd0-b332-049ea3c9e61b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:13.576886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59e71f-3617-4b0f-a73a-9005dcdc9e72 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:13.579111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:13.595131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa738f-d8be-4254-bd3d-dc9f1f07e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:13.595756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa738f-d8be-4254-bd3d-dc9f1f07e75d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:13.596029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa738f-d8be-4254-bd3d-dc9f1f07e75d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:13.596299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa738f-d8be-4254-bd3d-dc9f1f07e75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:13.606261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa738f-d8be-4254-bd3d-dc9f1f07e75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:13.607678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa738f-d8be-4254-bd3d-dc9f1f07e75d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:13.615848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b98b9-7c1d-4ff6-9263-3548966e806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:13.616473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b98b9-7c1d-4ff6-9263-3548966e806b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:13.616753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b98b9-7c1d-4ff6-9263-3548966e806b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:13.617020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b98b9-7c1d-4ff6-9263-3548966e806b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:13.617764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b98b9-7c1d-4ff6-9263-3548966e806b None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=024302e671b94c1ab0275621f3fdb02b, user.id=9f558f088da54d14b9c2d6a9010ae421) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:13.618068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b98b9-7c1d-4ff6-9263-3548966e806b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:13.619480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b98b9-7c1d-4ff6-9263-3548966e806b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-31fa738f-d8be-4254-bd3d-dc9f1f07e75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:10:13.622581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:13.623489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:10:13.624413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:10:13.624413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:13.624413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:13.624413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 536/1067] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:10:13.626986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 532/1068] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:10:13 2022] GET /identity/v3/role_assignments?user.id=9f558f088da54d14b9c2d6a9010ae421&scope.project.id=024302e671b94c1ab0275621f3fdb02b&role.id=36331ec562ff42f883fc883363bc4c8d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:13.629316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa738f-d8be-4254-bd3d-dc9f1f07e75d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:13.722313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:13.739422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa738f-d8be-4254-bd3d-dc9f1f07e75d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:13.775546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ad42b-c13b-4b34-be9b-3df54a1d492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:13.776223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ad42b-c13b-4b34-be9b-3df54a1d492a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:13.776501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ad42b-c13b-4b34-be9b-3df54a1d492a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:13.776766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ad42b-c13b-4b34-be9b-3df54a1d492a None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:13.777405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ad42b-c13b-4b34-be9b-3df54a1d492a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:13.777730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ad42b-c13b-4b34-be9b-3df54a1d492a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:13.779096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ad42b-c13b-4b34-be9b-3df54a1d492a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:13.797628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 537/1069] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:13.799279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b98b9-7c1d-4ff6-9263-3548966e806b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:13.910863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b98b9-7c1d-4ff6-9263-3548966e806b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:13.945337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca85661-ec1f-4a9d-bdc3-24967e6e233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:13.946009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca85661-ec1f-4a9d-bdc3-24967e6e233e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:13.946286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca85661-ec1f-4a9d-bdc3-24967e6e233e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:13.946554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca85661-ec1f-4a9d-bdc3-24967e6e233e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:13.954217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca85661-ec1f-4a9d-bdc3-24967e6e233e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:13.955661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca85661-ec1f-4a9d-bdc3-24967e6e233e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-9ca85661-ec1f-4a9d-bdc3-24967e6e233e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:10:13.966996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:13.967903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:13.968827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:10:13.968827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:10:13.968827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:10:13.968827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:10:13.968827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:10:13.968827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:10:13.968827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:10:13.968827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:13.972934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 533/1070] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:10:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:10:13.973942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ad42b-c13b-4b34-be9b-3df54a1d492a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:14.075123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ad42b-c13b-4b34-be9b-3df54a1d492a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:14.110534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89759e42-f6c5-4093-97a5-282a5a033b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:14.111141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89759e42-f6c5-4093-97a5-282a5a033b05 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:14.111419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89759e42-f6c5-4093-97a5-282a5a033b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:14.111687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89759e42-f6c5-4093-97a5-282a5a033b05 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:14.112370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89759e42-f6c5-4093-97a5-282a5a033b05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:14.112672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89759e42-f6c5-4093-97a5-282a5a033b05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:14.114122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89759e42-f6c5-4093-97a5-282a5a033b05 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:14.131198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 538/1071] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:10:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:14.132651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca85661-ec1f-4a9d-bdc3-24967e6e233e None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:14.162087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:14.236769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca85661-ec1f-4a9d-bdc3-24967e6e233e None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:14.271257 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049f44b-3793-4d75-a5f6-f064ce8dc708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:14.271919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049f44b-3793-4d75-a5f6-f064ce8dc708 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:14.272196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049f44b-3793-4d75-a5f6-f064ce8dc708 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:14.272471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049f44b-3793-4d75-a5f6-f064ce8dc708 None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/286f4e203e7a4a55bca0a58d061af2bd/roles/8d3762a5bc6a44cfb928f7da69fa6388` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:14.274621 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049f44b-3793-4d75-a5f6-f064ce8dc708 None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=286f4e203e7a4a55bca0a58d061af2bd, role_id=8d3762a5bc6a44cfb928f7da69fa6388)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:14.276063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049f44b-3793-4d75-a5f6-f064ce8dc708 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:14.289155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 534/1072] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:10:14 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/286f4e203e7a4a55bca0a58d061af2bd/roles/8d3762a5bc6a44cfb928f7da69fa6388 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:10:14.502995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a8d56-4e5f-4917-bc09-a7936a87e084 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:14.503553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a8d56-4e5f-4917-bc09-a7936a87e084 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:14.503553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a8d56-4e5f-4917-bc09-a7936a87e084 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:14.505002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 539/1073] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:14.510151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f19a6-14d6-428c-bd05-1f9ea5bcb86c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:14.510527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f19a6-14d6-428c-bd05-1f9ea5bcb86c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:14.510721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f19a6-14d6-428c-bd05-1f9ea5bcb86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:14.528611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7e0f19a6-14d6-428c-bd05-1f9ea5bcb86c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:14.529798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7e0f19a6-14d6-428c-bd05-1f9ea5bcb86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:14.595686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 535/1074] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:14.879251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a8d56-4e5f-4917-bc09-a7936a87e084 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:14.886099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:14.993942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a8d56-4e5f-4917-bc09-a7936a87e084 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:15.029851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5f0e8-c471-4dd9-92f2-a886b6eea113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:15.030463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5f0e8-c471-4dd9-92f2-a886b6eea113 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:15.033537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5f0e8-c471-4dd9-92f2-a886b6eea113 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:15.033537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5f0e8-c471-4dd9-92f2-a886b6eea113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:15.102282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5f0e8-c471-4dd9-92f2-a886b6eea113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:15.103715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5f0e8-c471-4dd9-92f2-a886b6eea113 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:15.106344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:15.204668 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 540/1075] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:10:15.210300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f19a6-14d6-428c-bd05-1f9ea5bcb86c None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:15.228020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:15.310493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f19a6-14d6-428c-bd05-1f9ea5bcb86c None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:15.344576 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851a9b0-37e5-4779-9fab-3a11f1a42c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:15.345200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851a9b0-37e5-4779-9fab-3a11f1a42c14 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:15.345469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851a9b0-37e5-4779-9fab-3a11f1a42c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:15.345759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851a9b0-37e5-4779-9fab-3a11f1a42c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:15.355534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851a9b0-37e5-4779-9fab-3a11f1a42c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:15.356962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851a9b0-37e5-4779-9fab-3a11f1a42c14 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-4851a9b0-37e5-4779-9fab-3a11f1a42c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:10:15.371125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:15.372125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:10:15.373078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:10:15.373078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:15.373078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:15.377544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 536/1076] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:15.378346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5f0e8-c471-4dd9-92f2-a886b6eea113 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:15.476362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5f0e8-c471-4dd9-92f2-a886b6eea113 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:15.478598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:15.511462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c118626-20cb-402e-b51d-8056aa2411aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:15.512080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118626-20cb-402e-b51d-8056aa2411aa None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:15.512330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118626-20cb-402e-b51d-8056aa2411aa None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:15.512587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118626-20cb-402e-b51d-8056aa2411aa None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:15.513215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c118626-20cb-402e-b51d-8056aa2411aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:15.513528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c118626-20cb-402e-b51d-8056aa2411aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:15.515482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c118626-20cb-402e-b51d-8056aa2411aa None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:15.529035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 541/1077] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:15.776218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5e323-cc22-41b3-8d37-f2b5cf8451f8 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:15.776698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5e323-cc22-41b3-8d37-f2b5cf8451f8 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:15.776698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5e323-cc22-41b3-8d37-f2b5cf8451f8 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:15.778257 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 537/1078] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:15.784246 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519b000-fd62-4b2c-b7e1-eae5905e3a77 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:15.784539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519b000-fd62-4b2c-b7e1-eae5905e3a77 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:15.784812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519b000-fd62-4b2c-b7e1-eae5905e3a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:15.803132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-b519b000-fd62-4b2c-b7e1-eae5905e3a77 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:15.804314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b519b000-fd62-4b2c-b7e1-eae5905e3a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:15.877484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 542/1079] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:16.149620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5e323-cc22-41b3-8d37-f2b5cf8451f8 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:16.215113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:16.249133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5e323-cc22-41b3-8d37-f2b5cf8451f8 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:16.282609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74154b3d-fd94-4370-9ec6-5e446ee2d299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:16.283221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74154b3d-fd94-4370-9ec6-5e446ee2d299 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:16.283490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74154b3d-fd94-4370-9ec6-5e446ee2d299 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:16.283756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74154b3d-fd94-4370-9ec6-5e446ee2d299 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:16.293463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74154b3d-fd94-4370-9ec6-5e446ee2d299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-74154b3d-fd94-4370-9ec6-5e446ee2d299 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:10:16.298184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:16.299270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:16.300125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 538/1080] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:10:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:16.305625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519b000-fd62-4b2c-b7e1-eae5905e3a77 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:16.401129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:16.408811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519b000-fd62-4b2c-b7e1-eae5905e3a77 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:16.442330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558940-4e86-4b29-9bae-e77c3b17b86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:16.442933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558940-4e86-4b29-9bae-e77c3b17b86d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:16.443210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558940-4e86-4b29-9bae-e77c3b17b86d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:16.443483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558940-4e86-4b29-9bae-e77c3b17b86d None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:16.444138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558940-4e86-4b29-9bae-e77c3b17b86d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:16.444438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558940-4e86-4b29-9bae-e77c3b17b86d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:16.445811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558940-4e86-4b29-9bae-e77c3b17b86d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:16.455973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 543/1081] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:10:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:16.462046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74154b3d-fd94-4370-9ec6-5e446ee2d299 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:16.486520 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e945f9-521d-44ed-9baf-142c1a2baee8 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:16.486796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e945f9-521d-44ed-9baf-142c1a2baee8 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:16.487061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e945f9-521d-44ed-9baf-142c1a2baee8 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:16.488371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 544/1082] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:10:16.493282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b146-4a04-4476-b5bc-396fbccffdb6 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:16.493587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b146-4a04-4476-b5bc-396fbccffdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:16.493858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b146-4a04-4476-b5bc-396fbccffdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:16.511281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-57f1b146-4a04-4476-b5bc-396fbccffdb6 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:16.512450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-57f1b146-4a04-4476-b5bc-396fbccffdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:16.562135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74154b3d-fd94-4370-9ec6-5e446ee2d299 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:16.576527 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 545/1083] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:10:16.588703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:16.597623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592611f2-d9c6-4128-acaf-a453a1be5447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:16.598243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592611f2-d9c6-4128-acaf-a453a1be5447 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:16.598515 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592611f2-d9c6-4128-acaf-a453a1be5447 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:16.598779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592611f2-d9c6-4128-acaf-a453a1be5447 None admin] PATH_INFO: `/v3/users/9f558f088da54d14b9c2d6a9010ae421` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:16.599867 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592611f2-d9c6-4128-acaf-a453a1be5447 None admin] RBAC: Authorizing `identity:get_user(user_id=9f558f088da54d14b9c2d6a9010ae421)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:16.601284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592611f2-d9c6-4128-acaf-a453a1be5447 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:16.602963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 539/1084] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:16 2022] GET /identity/v3/users/9f558f088da54d14b9c2d6a9010ae421 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:16.608656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b146-4a04-4476-b5bc-396fbccffdb6 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:16.687982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:16.703440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b146-4a04-4476-b5bc-396fbccffdb6 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:16.735628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe31bea-ba10-46ee-922e-a41b46951870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:16.736245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe31bea-ba10-46ee-922e-a41b46951870 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:16.736517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe31bea-ba10-46ee-922e-a41b46951870 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:16.736781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe31bea-ba10-46ee-922e-a41b46951870 None admin] PATH_INFO: `/v3/projects/024302e671b94c1ab0275621f3fdb02b` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:16.737890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe31bea-ba10-46ee-922e-a41b46951870 None admin] RBAC: Authorizing `identity:get_project(project_id=024302e671b94c1ab0275621f3fdb02b)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:16.739294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe31bea-ba10-46ee-922e-a41b46951870 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:16.740876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 546/1085] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:10:16 2022] GET /identity/v3/projects/024302e671b94c1ab0275621f3fdb02b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:16.746744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592611f2-d9c6-4128-acaf-a453a1be5447 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:16.844102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe31bea-ba10-46ee-922e-a41b46951870 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:16.844327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592611f2-d9c6-4128-acaf-a453a1be5447 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:16.879025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb1cde-88b3-4ed1-b921-88f6bd5b5f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:16.879674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb1cde-88b3-4ed1-b921-88f6bd5b5f40 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:16.879948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb1cde-88b3-4ed1-b921-88f6bd5b5f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:16.880307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb1cde-88b3-4ed1-b921-88f6bd5b5f40 None admin] PATH_INFO: `/v3/projects/024302e671b94c1ab0275621f3fdb02b/users/9f558f088da54d14b9c2d6a9010ae421/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:16.882110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb1cde-88b3-4ed1-b921-88f6bd5b5f40 None admin] RBAC: Authorizing `identity:create_grant(project_id=024302e671b94c1ab0275621f3fdb02b, user_id=9f558f088da54d14b9c2d6a9010ae421, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:16.883531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb1cde-88b3-4ed1-b921-88f6bd5b5f40 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:16.897344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 540/1086] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:10:16 2022] PUT /identity/v3/projects/024302e671b94c1ab0275621f3fdb02b/users/9f558f088da54d14b9c2d6a9010ae421/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:10:16.952306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe31bea-ba10-46ee-922e-a41b46951870 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:16.988636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebea292-62c5-42db-b0a6-fad08c302a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:16.989291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebea292-62c5-42db-b0a6-fad08c302a8f None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:16.989713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebea292-62c5-42db-b0a6-fad08c302a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:16.989990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebea292-62c5-42db-b0a6-fad08c302a8f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:17.001904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebea292-62c5-42db-b0a6-fad08c302a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-9ebea292-62c5-42db-b0a6-fad08c302a8f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:10:17.006749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:17.007847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:17.008731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 547/1087] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 20:10:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:10:17.014314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb1cde-88b3-4ed1-b921-88f6bd5b5f40 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:17.073700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:17.118451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb1cde-88b3-4ed1-b921-88f6bd5b5f40 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:17.152628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acaa78-51d2-4497-b588-76f8e7a39a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:17.153223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acaa78-51d2-4497-b588-76f8e7a39a84 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:17.153492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acaa78-51d2-4497-b588-76f8e7a39a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:17.153786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acaa78-51d2-4497-b588-76f8e7a39a84 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:17.154435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acaa78-51d2-4497-b588-76f8e7a39a84 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:17.154730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acaa78-51d2-4497-b588-76f8e7a39a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:17.156103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acaa78-51d2-4497-b588-76f8e7a39a84 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:17.163767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 541/1088] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 20:10:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:17.169667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebea292-62c5-42db-b0a6-fad08c302a8f None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:17.185654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:17.269488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebea292-62c5-42db-b0a6-fad08c302a8f None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:17.302676 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2f13b-89b5-4337-8196-2da038acbc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:17.303294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2f13b-89b5-4337-8196-2da038acbc64 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:17.303570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2f13b-89b5-4337-8196-2da038acbc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:17.303840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2f13b-89b5-4337-8196-2da038acbc64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:17.314180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2f13b-89b5-4337-8196-2da038acbc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:17.315860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2f13b-89b5-4337-8196-2da038acbc64 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-fae2f13b-89b5-4337-8196-2da038acbc64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:10:17.330026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:17.331158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:10:17.332140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:10:17.332140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:17.332140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:17.332140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 548/1089] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:17.336956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acaa78-51d2-4497-b588-76f8e7a39a84 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:17.434502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acaa78-51d2-4497-b588-76f8e7a39a84 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:17.451358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:17.468488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e2df0-d5ce-41eb-8fa3-7bbada159c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:17.469085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e2df0-d5ce-41eb-8fa3-7bbada159c25 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:17.469359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e2df0-d5ce-41eb-8fa3-7bbada159c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:17.469658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e2df0-d5ce-41eb-8fa3-7bbada159c25 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:17.470293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e2df0-d5ce-41eb-8fa3-7bbada159c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:17.470593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e2df0-d5ce-41eb-8fa3-7bbada159c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:17.471942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e2df0-d5ce-41eb-8fa3-7bbada159c25 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:17.483126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 542/1090] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:10:17.489295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2f13b-89b5-4337-8196-2da038acbc64 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:17.553568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:17.588490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2f13b-89b5-4337-8196-2da038acbc64 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:17.621390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f868f-54dd-44f4-8d5b-893efbc8dfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:17.622050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f868f-54dd-44f4-8d5b-893efbc8dfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:17.622326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f868f-54dd-44f4-8d5b-893efbc8dfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:17.622593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f868f-54dd-44f4-8d5b-893efbc8dfb3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:17.630432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f868f-54dd-44f4-8d5b-893efbc8dfb3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:17.631855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f868f-54dd-44f4-8d5b-893efbc8dfb3 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-6c2f868f-54dd-44f4-8d5b-893efbc8dfb3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:10:17.642682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:17.643754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:17.644712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 549/1091] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 20:10:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:10:17.649543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e2df0-d5ce-41eb-8fa3-7bbada159c25 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:17.746927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e2df0-d5ce-41eb-8fa3-7bbada159c25 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:17.782403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16300278-393e-4e72-b16c-37a18edb1a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:17.783007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16300278-393e-4e72-b16c-37a18edb1a3b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:17.783288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16300278-393e-4e72-b16c-37a18edb1a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:17.783561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16300278-393e-4e72-b16c-37a18edb1a3b None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:17.784278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16300278-393e-4e72-b16c-37a18edb1a3b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:17.784585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16300278-393e-4e72-b16c-37a18edb1a3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:17.786046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16300278-393e-4e72-b16c-37a18edb1a3b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:17.803225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 543/1092] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 20:10:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:17.809286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f868f-54dd-44f4-8d5b-893efbc8dfb3 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:17.914991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f868f-54dd-44f4-8d5b-893efbc8dfb3 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:17.939719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:17.948506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199e841-a835-468d-96e2-b3f887b1c9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:17.949107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199e841-a835-468d-96e2-b3f887b1c9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:17.949385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199e841-a835-468d-96e2-b3f887b1c9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:17.949697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199e841-a835-468d-96e2-b3f887b1c9c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:18.014490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199e841-a835-468d-96e2-b3f887b1c9c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:18.016459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199e841-a835-468d-96e2-b3f887b1c9c4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:18.112831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 550/1093] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:10:18.118567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16300278-393e-4e72-b16c-37a18edb1a3b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:18.152772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:18.223268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16300278-393e-4e72-b16c-37a18edb1a3b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:18.256445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8396b30-fc08-4972-8cdc-b9e49c40ae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:18.257067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8396b30-fc08-4972-8cdc-b9e49c40ae54 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:18.257346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8396b30-fc08-4972-8cdc-b9e49c40ae54 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:18.257649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8396b30-fc08-4972-8cdc-b9e49c40ae54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:18.267268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8396b30-fc08-4972-8cdc-b9e49c40ae54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:18.268704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8396b30-fc08-4972-8cdc-b9e49c40ae54 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-f8396b30-fc08-4972-8cdc-b9e49c40ae54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:10:18.282796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:18.283738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:10:18.284609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:10:18.284609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:10:18.284609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:18.284609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 544/1094] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:18.289685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199e841-a835-468d-96e2-b3f887b1c9c4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:18.316957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:18.387589 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199e841-a835-468d-96e2-b3f887b1c9c4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:18.420474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeeeec-cdf4-4491-a0fc-32998318dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:18.421635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeeeec-cdf4-4491-a0fc-32998318dd7a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:18.421933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeeeec-cdf4-4491-a0fc-32998318dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:18.422203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeeeec-cdf4-4491-a0fc-32998318dd7a None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:18.422836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeeeec-cdf4-4491-a0fc-32998318dd7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:18.423137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeeeec-cdf4-4491-a0fc-32998318dd7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:18.424485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeeeec-cdf4-4491-a0fc-32998318dd7a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:18.435498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 551/1095] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:18.441479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8396b30-fc08-4972-8cdc-b9e49c40ae54 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:18.527173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:18.540225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8396b30-fc08-4972-8cdc-b9e49c40ae54 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:18.574744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94836ef-24e6-4b38-b550-b44f6e77e69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:18.575368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94836ef-24e6-4b38-b550-b44f6e77e69c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:18.575644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94836ef-24e6-4b38-b550-b44f6e77e69c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:18.575913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94836ef-24e6-4b38-b550-b44f6e77e69c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:18.582382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94836ef-24e6-4b38-b550-b44f6e77e69c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:18.583814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94836ef-24e6-4b38-b550-b44f6e77e69c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-e94836ef-24e6-4b38-b550-b44f6e77e69c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:10:18.595240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:18.596259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:18.597187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:10:18.597187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:10:18.597187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:10:18.597187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:10:18.597187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:10:18.597187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:10:18.597187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:10:18.597187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:18.597187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 545/1096] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:10:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:18.602385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeeeec-cdf4-4491-a0fc-32998318dd7a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:18.685558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:18.701147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeeeec-cdf4-4491-a0fc-32998318dd7a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:18.734434 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d8682-2e26-408e-b361-600c778416ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:18.735022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d8682-2e26-408e-b361-600c778416ca None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:18.735300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d8682-2e26-408e-b361-600c778416ca None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:18.735571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d8682-2e26-408e-b361-600c778416ca None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:18.736252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d8682-2e26-408e-b361-600c778416ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:18.736559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d8682-2e26-408e-b361-600c778416ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:18.738001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d8682-2e26-408e-b361-600c778416ca None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:18.750354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 552/1097] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:10:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:18.756387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94836ef-24e6-4b38-b550-b44f6e77e69c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:18.852172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94836ef-24e6-4b38-b550-b44f6e77e69c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:18.887218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924fa21-5d92-4aca-ae83-1dc01167792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:18.887818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924fa21-5d92-4aca-ae83-1dc01167792e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:18.888096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924fa21-5d92-4aca-ae83-1dc01167792e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:18.888363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924fa21-5d92-4aca-ae83-1dc01167792e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:18.889077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924fa21-5d92-4aca-ae83-1dc01167792e None admin] RBAC: Adding query filter params (role.id=8d3762a5bc6a44cfb928f7da69fa6388, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=286f4e203e7a4a55bca0a58d061af2bd) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:18.889383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924fa21-5d92-4aca-ae83-1dc01167792e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:18.891375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924fa21-5d92-4aca-ae83-1dc01167792e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:18.897668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 546/1098] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:10:18 2022] GET /identity/v3/role_assignments?user.id=286f4e203e7a4a55bca0a58d061af2bd&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=8d3762a5bc6a44cfb928f7da69fa6388 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:19.026063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468ab05-c0da-4fc7-9283-18b3d317357b None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:19.026265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468ab05-c0da-4fc7-9283-18b3d317357b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:19.026552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468ab05-c0da-4fc7-9283-18b3d317357b None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:19.027900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 553/1099] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:19.033638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db983a0e-a1e2-4f10-8350-2c7d216f7bed None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:19.033921 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db983a0e-a1e2-4f10-8350-2c7d216f7bed None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:19.034186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db983a0e-a1e2-4f10-8350-2c7d216f7bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:19.053454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-db983a0e-a1e2-4f10-8350-2c7d216f7bed None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:19.054708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-db983a0e-a1e2-4f10-8350-2c7d216f7bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:19.063544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:19.124737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 547/1100] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:19.395061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468ab05-c0da-4fc7-9283-18b3d317357b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:19.497995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468ab05-c0da-4fc7-9283-18b3d317357b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:19.532368 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd7cb8-f993-4503-8ff3-c71ca34f8201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:19.532998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd7cb8-f993-4503-8ff3-c71ca34f8201 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:19.533276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd7cb8-f993-4503-8ff3-c71ca34f8201 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:19.533580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd7cb8-f993-4503-8ff3-c71ca34f8201 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:19.543761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd7cb8-f993-4503-8ff3-c71ca34f8201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-66fd7cb8-f993-4503-8ff3-c71ca34f8201 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:10:19.548540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:19.549425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:19.550277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 554/1101] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:10:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:19.555818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db983a0e-a1e2-4f10-8350-2c7d216f7bed None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:19.658397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db983a0e-a1e2-4f10-8350-2c7d216f7bed None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:19.691530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a72fc-796e-47bb-b4e3-86f243888bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:19.692132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a72fc-796e-47bb-b4e3-86f243888bbe None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:19.692408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a72fc-796e-47bb-b4e3-86f243888bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:19.692674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a72fc-796e-47bb-b4e3-86f243888bbe None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:19.693337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a72fc-796e-47bb-b4e3-86f243888bbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:19.693664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a72fc-796e-47bb-b4e3-86f243888bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:19.695023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a72fc-796e-47bb-b4e3-86f243888bbe None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:19.703644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 548/1102] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:10:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:19.709668 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd7cb8-f993-4503-8ff3-c71ca34f8201 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:19.719756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:19.811467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd7cb8-f993-4503-8ff3-c71ca34f8201 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:19.845174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d24ab-d0dd-4a19-9ec8-50f414202828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:19.845821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d24ab-d0dd-4a19-9ec8-50f414202828 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:19.846097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d24ab-d0dd-4a19-9ec8-50f414202828 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:19.846367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d24ab-d0dd-4a19-9ec8-50f414202828 None admin] PATH_INFO: `/v3/users/9f558f088da54d14b9c2d6a9010ae421` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:19.847468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d24ab-d0dd-4a19-9ec8-50f414202828 None admin] RBAC: Authorizing `identity:get_user(user_id=9f558f088da54d14b9c2d6a9010ae421)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:19.848883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d24ab-d0dd-4a19-9ec8-50f414202828 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:19.850480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 555/1103] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:19 2022] GET /identity/v3/users/9f558f088da54d14b9c2d6a9010ae421 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:19.856289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a72fc-796e-47bb-b4e3-86f243888bbe None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:19.860058 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:19.953586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a72fc-796e-47bb-b4e3-86f243888bbe None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:19.984638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53e1a2-c344-418f-aa70-954f3f0839b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:19.985224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53e1a2-c344-418f-aa70-954f3f0839b2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:19.985495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53e1a2-c344-418f-aa70-954f3f0839b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:19.985918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53e1a2-c344-418f-aa70-954f3f0839b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:20.049373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53e1a2-c344-418f-aa70-954f3f0839b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:20.050843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53e1a2-c344-418f-aa70-954f3f0839b2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:20.053279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:20.145505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 549/1104] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:10:20.151509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d24ab-d0dd-4a19-9ec8-50f414202828 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:20.224222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:20.248163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d24ab-d0dd-4a19-9ec8-50f414202828 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:20.281304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0b59b-2664-4e4e-bf58-4675ff434565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:20.281946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b59b-2664-4e4e-bf58-4675ff434565 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:20.282227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b59b-2664-4e4e-bf58-4675ff434565 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:20.282502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b59b-2664-4e4e-bf58-4675ff434565 None admin] PATH_INFO: `/v3/projects/024302e671b94c1ab0275621f3fdb02b` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:20.283601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0b59b-2664-4e4e-bf58-4675ff434565 None admin] RBAC: Authorizing `identity:get_project(project_id=024302e671b94c1ab0275621f3fdb02b)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:20.285024 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0b59b-2664-4e4e-bf58-4675ff434565 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:20.286650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 556/1105] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:10:20 2022] GET /identity/v3/projects/024302e671b94c1ab0275621f3fdb02b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:20.292587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53e1a2-c344-418f-aa70-954f3f0839b2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:20.387609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53e1a2-c344-418f-aa70-954f3f0839b2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:20.389874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:20.420583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8b907-27d9-4d11-a7de-5ac9190c19ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:20.421785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8b907-27d9-4d11-a7de-5ac9190c19ca None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:20.422057 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8b907-27d9-4d11-a7de-5ac9190c19ca None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:20.422321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8b907-27d9-4d11-a7de-5ac9190c19ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:20.423022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8b907-27d9-4d11-a7de-5ac9190c19ca None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=024302e671b94c1ab0275621f3fdb02b, user.id=9f558f088da54d14b9c2d6a9010ae421) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:20.423322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8b907-27d9-4d11-a7de-5ac9190c19ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:20.424722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8b907-27d9-4d11-a7de-5ac9190c19ca None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:20.430290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 550/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:10:20 2022] GET /identity/v3/role_assignments?user.id=9f558f088da54d14b9c2d6a9010ae421&scope.project.id=024302e671b94c1ab0275621f3fdb02b&role.id=36331ec562ff42f883fc883363bc4c8d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:22.576684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30032632-9ab5-43ed-941d-142b7f609b64 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:22.577237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30032632-9ab5-43ed-941d-142b7f609b64 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:22.577237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30032632-9ab5-43ed-941d-142b7f609b64 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:22.578623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 557/1107] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:22.583852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc991d2-a2dd-49d5-982f-9c45710a535a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:22.584244 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc991d2-a2dd-49d5-982f-9c45710a535a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:22.584391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc991d2-a2dd-49d5-982f-9c45710a535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:22.602094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-4fc991d2-a2dd-49d5-982f-9c45710a535a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:22.603247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4fc991d2-a2dd-49d5-982f-9c45710a535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:22.665850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 551/1108] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:22.939238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30032632-9ab5-43ed-941d-142b7f609b64 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:23.044927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30032632-9ab5-43ed-941d-142b7f609b64 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:23.078511 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa57b9-96c3-44a9-a753-fd21599afb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:23.079122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa57b9-96c3-44a9-a753-fd21599afb2c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:23.079396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa57b9-96c3-44a9-a753-fd21599afb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:23.079663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa57b9-96c3-44a9-a753-fd21599afb2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:23.080744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa57b9-96c3-44a9-a753-fd21599afb2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:23.082189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa57b9-96c3-44a9-a753-fd21599afb2c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:23.083727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 558/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:23.089393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc991d2-a2dd-49d5-982f-9c45710a535a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:23.179255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:23.183259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc991d2-a2dd-49d5-982f-9c45710a535a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:23.216985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5fdbf-bc35-428b-972c-df55b3bce54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:23.217597 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5fdbf-bc35-428b-972c-df55b3bce54c None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:23.218505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5fdbf-bc35-428b-972c-df55b3bce54c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:23.218812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5fdbf-bc35-428b-972c-df55b3bce54c None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:23.219447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5fdbf-bc35-428b-972c-df55b3bce54c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:23.220858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5fdbf-bc35-428b-972c-df55b3bce54c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:23.241841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 552/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:10:23 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:10:25.412216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bc85b-e6b0-4af0-a0b2-7e89c99d4a85 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:25.412757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bc85b-e6b0-4af0-a0b2-7e89c99d4a85 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:25.412757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bc85b-e6b0-4af0-a0b2-7e89c99d4a85 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:25.414125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 559/1111] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:25.419357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7e1cc-2710-41c8-9ea2-9a2bfb7db797 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:25.419784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7e1cc-2710-41c8-9ea2-9a2bfb7db797 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:25.419885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7e1cc-2710-41c8-9ea2-9a2bfb7db797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:25.438215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-72c7e1cc-2710-41c8-9ea2-9a2bfb7db797 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:25.439648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-72c7e1cc-2710-41c8-9ea2-9a2bfb7db797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:25.503210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 553/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:25.771497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bc85b-e6b0-4af0-a0b2-7e89c99d4a85 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:25.784244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:25.870359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bc85b-e6b0-4af0-a0b2-7e89c99d4a85 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:25.903010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a7600-a304-40d9-bd02-c72c6d3aa2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:25.903624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a7600-a304-40d9-bd02-c72c6d3aa2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:25.903896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a7600-a304-40d9-bd02-c72c6d3aa2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:25.904162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a7600-a304-40d9-bd02-c72c6d3aa2d4 None admin] PATH_INFO: `/v3/roles/6a600852dc8442fdb2ffc3d663e274ff` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:25.906133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a7600-a304-40d9-bd02-c72c6d3aa2d4 None admin] RBAC: Authorizing `identity:get_role(role_id=6a600852dc8442fdb2ffc3d663e274ff)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:25.907476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a7600-a304-40d9-bd02-c72c6d3aa2d4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:25.908770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 560/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:25 2022] GET /identity/v3/roles/6a600852dc8442fdb2ffc3d663e274ff => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:25.914555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7e1cc-2710-41c8-9ea2-9a2bfb7db797 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:25.993885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:26.010297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7e1cc-2710-41c8-9ea2-9a2bfb7db797 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:26.043172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fe120-1c0f-4e58-a243-257e26c19e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:26.043724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fe120-1c0f-4e58-a243-257e26c19e9e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:26.043998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fe120-1c0f-4e58-a243-257e26c19e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:26.044262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fe120-1c0f-4e58-a243-257e26c19e9e None admin] PATH_INFO: `/v3/users/94bb3d71f58b48b684498cff082f5ce0` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:26.056774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fe120-1c0f-4e58-a243-257e26c19e9e None admin] RBAC: Authorizing `identity:get_user(user_id=94bb3d71f58b48b684498cff082f5ce0)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:26.058231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fe120-1c0f-4e58-a243-257e26c19e9e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:26.059802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 554/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:25 2022] GET /identity/v3/users/94bb3d71f58b48b684498cff082f5ce0 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:26.066340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a7600-a304-40d9-bd02-c72c6d3aa2d4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:26.141954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:26.163541 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a7600-a304-40d9-bd02-c72c6d3aa2d4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:26.196358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5490ed7-6fae-4eca-9204-16b6c7c3ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:26.196942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5490ed7-6fae-4eca-9204-16b6c7c3ba96 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:26.197212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5490ed7-6fae-4eca-9204-16b6c7c3ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:26.197475 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5490ed7-6fae-4eca-9204-16b6c7c3ba96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:26.261433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5490ed7-6fae-4eca-9204-16b6c7c3ba96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:26.262869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5490ed7-6fae-4eca-9204-16b6c7c3ba96 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:26.335603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:26.356757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 561/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:10:26.363213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fe120-1c0f-4e58-a243-257e26c19e9e None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:26.458435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fe120-1c0f-4e58-a243-257e26c19e9e None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:26.492561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6cac9-c401-4def-9d60-a3c326b638a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:26.493163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6cac9-c401-4def-9d60-a3c326b638a4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:26.493435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6cac9-c401-4def-9d60-a3c326b638a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:26.493726 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6cac9-c401-4def-9d60-a3c326b638a4 None admin] PATH_INFO: `/v3/projects/024302e671b94c1ab0275621f3fdb02b` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:26.494810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6cac9-c401-4def-9d60-a3c326b638a4 None admin] RBAC: Authorizing `identity:get_project(project_id=024302e671b94c1ab0275621f3fdb02b)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:26.496221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6cac9-c401-4def-9d60-a3c326b638a4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:26.497831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 555/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:10:26 2022] GET /identity/v3/projects/024302e671b94c1ab0275621f3fdb02b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:26.503911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5490ed7-6fae-4eca-9204-16b6c7c3ba96 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:26.597947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5490ed7-6fae-4eca-9204-16b6c7c3ba96 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:26.629774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3b116-5433-4df8-801e-bae01f8bf895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:26.630358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3b116-5433-4df8-801e-bae01f8bf895 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:26.630632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3b116-5433-4df8-801e-bae01f8bf895 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:26.630899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3b116-5433-4df8-801e-bae01f8bf895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:26.631604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3b116-5433-4df8-801e-bae01f8bf895 None admin] RBAC: Adding query filter params (role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=024302e671b94c1ab0275621f3fdb02b, user.id=94bb3d71f58b48b684498cff082f5ce0) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:26.631906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3b116-5433-4df8-801e-bae01f8bf895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:26.633300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3b116-5433-4df8-801e-bae01f8bf895 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:26.638418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 562/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:10:26 2022] GET /identity/v3/role_assignments?user.id=94bb3d71f58b48b684498cff082f5ce0&scope.project.id=024302e671b94c1ab0275621f3fdb02b&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:28.715964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d13b2-9bda-4672-9db9-b4bb9e222c1d None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:28.716513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d13b2-9bda-4672-9db9-b4bb9e222c1d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:28.716513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d13b2-9bda-4672-9db9-b4bb9e222c1d None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:28.717916 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 556/1118] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:28.723195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92016c10-5b50-490e-b3e4-72b87290b395 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:28.723585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92016c10-5b50-490e-b3e4-72b87290b395 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:28.723731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92016c10-5b50-490e-b3e4-72b87290b395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:28.741836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-92016c10-5b50-490e-b3e4-72b87290b395 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:28.742993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-92016c10-5b50-490e-b3e4-72b87290b395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:28.776713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:28.807303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 563/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:29.075565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d13b2-9bda-4672-9db9-b4bb9e222c1d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:29.088199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:29.173298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d13b2-9bda-4672-9db9-b4bb9e222c1d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:29.206387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2358bd8-549d-4b85-af60-c9580d3083b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:29.206996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358bd8-549d-4b85-af60-c9580d3083b7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:29.207274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358bd8-549d-4b85-af60-c9580d3083b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:29.207544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2358bd8-549d-4b85-af60-c9580d3083b7 None admin] PATH_INFO: `/v3/roles/6a600852dc8442fdb2ffc3d663e274ff` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:29.208899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2358bd8-549d-4b85-af60-c9580d3083b7 None admin] RBAC: Authorizing `identity:get_role(role_id=6a600852dc8442fdb2ffc3d663e274ff)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:29.210277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2358bd8-549d-4b85-af60-c9580d3083b7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:29.211561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 557/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:29 2022] GET /identity/v3/roles/6a600852dc8442fdb2ffc3d663e274ff => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:29.217187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92016c10-5b50-490e-b3e4-72b87290b395 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:29.313712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92016c10-5b50-490e-b3e4-72b87290b395 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:29.346904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b0820-7220-415c-a4e2-a949ddb31ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:29.347519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b0820-7220-415c-a4e2-a949ddb31ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:29.347797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b0820-7220-415c-a4e2-a949ddb31ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:29.348071 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b0820-7220-415c-a4e2-a949ddb31ce9 None admin] PATH_INFO: `/v3/users/94bb3d71f58b48b684498cff082f5ce0` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:29.349165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b0820-7220-415c-a4e2-a949ddb31ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=94bb3d71f58b48b684498cff082f5ce0)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:29.350608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b0820-7220-415c-a4e2-a949ddb31ce9 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:29.352149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 564/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:29 2022] GET /identity/v3/users/94bb3d71f58b48b684498cff082f5ce0 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:29.357613 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2358bd8-549d-4b85-af60-c9580d3083b7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:29.431967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:29.453517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2358bd8-549d-4b85-af60-c9580d3083b7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:29.485542 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838404fa-a648-4d2d-b184-a22ec5e44ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:29.486157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838404fa-a648-4d2d-b184-a22ec5e44ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:29.486434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838404fa-a648-4d2d-b184-a22ec5e44ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:29.486704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838404fa-a648-4d2d-b184-a22ec5e44ed1 None admin] PATH_INFO: `/v3/projects/024302e671b94c1ab0275621f3fdb02b` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:29.487793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838404fa-a648-4d2d-b184-a22ec5e44ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=024302e671b94c1ab0275621f3fdb02b)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:29.489212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838404fa-a648-4d2d-b184-a22ec5e44ed1 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:29.490842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 558/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:10:29 2022] GET /identity/v3/projects/024302e671b94c1ab0275621f3fdb02b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:29.496522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b0820-7220-415c-a4e2-a949ddb31ce9 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:29.537081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:29.592177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b0820-7220-415c-a4e2-a949ddb31ce9 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:29.625022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9f1be-317c-4952-b903-26b70cf2f4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:29.625619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9f1be-317c-4952-b903-26b70cf2f4c7 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:29.625891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9f1be-317c-4952-b903-26b70cf2f4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:29.626155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9f1be-317c-4952-b903-26b70cf2f4c7 None admin] PATH_INFO: `/v3/projects/024302e671b94c1ab0275621f3fdb02b/users/94bb3d71f58b48b684498cff082f5ce0/roles/6a600852dc8442fdb2ffc3d663e274ff` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:29.628215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9f1be-317c-4952-b903-26b70cf2f4c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=024302e671b94c1ab0275621f3fdb02b, user_id=94bb3d71f58b48b684498cff082f5ce0, role_id=6a600852dc8442fdb2ffc3d663e274ff)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:29.629634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9f1be-317c-4952-b903-26b70cf2f4c7 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:29.641740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 565/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:10:29 2022] PUT /identity/v3/projects/024302e671b94c1ab0275621f3fdb02b/users/94bb3d71f58b48b684498cff082f5ce0/roles/6a600852dc8442fdb2ffc3d663e274ff => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:10:31.735777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7f615-6b65-4c52-ab9c-d2e7a2ec5339 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:31.736320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7f615-6b65-4c52-ab9c-d2e7a2ec5339 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:31.736320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf7f615-6b65-4c52-ab9c-d2e7a2ec5339 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:31.737678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 559/1124] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:31.742920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98336221-e597-4be2-b9bb-86e5ac056ef2 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:31.743308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98336221-e597-4be2-b9bb-86e5ac056ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:31.743473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98336221-e597-4be2-b9bb-86e5ac056ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:31.761302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-98336221-e597-4be2-b9bb-86e5ac056ef2 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:31.762511 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-98336221-e597-4be2-b9bb-86e5ac056ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:31.826007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 566/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:32.093354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7f615-6b65-4c52-ab9c-d2e7a2ec5339 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:32.190374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf7f615-6b65-4c52-ab9c-d2e7a2ec5339 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:32.222356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60b230-b675-40ae-98da-f5128647cdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:32.222960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60b230-b675-40ae-98da-f5128647cdf0 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:32.223231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60b230-b675-40ae-98da-f5128647cdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:32.223495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60b230-b675-40ae-98da-f5128647cdf0 None admin] PATH_INFO: `/v3/roles/6a600852dc8442fdb2ffc3d663e274ff` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:32.224838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60b230-b675-40ae-98da-f5128647cdf0 None admin] RBAC: Authorizing `identity:get_role(role_id=6a600852dc8442fdb2ffc3d663e274ff)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:32.226213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60b230-b675-40ae-98da-f5128647cdf0 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:32.227491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 560/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:32 2022] GET /identity/v3/roles/6a600852dc8442fdb2ffc3d663e274ff => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:32.233073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98336221-e597-4be2-b9bb-86e5ac056ef2 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:32.262044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:32.331230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98336221-e597-4be2-b9bb-86e5ac056ef2 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:32.363604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c1efa-d5cf-43ae-96a0-32421e1da567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:32.364157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c1efa-d5cf-43ae-96a0-32421e1da567 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:32.364429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c1efa-d5cf-43ae-96a0-32421e1da567 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:32.364698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c1efa-d5cf-43ae-96a0-32421e1da567 None admin] PATH_INFO: `/v3/users/94bb3d71f58b48b684498cff082f5ce0` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:32.365805 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c1efa-d5cf-43ae-96a0-32421e1da567 None admin] RBAC: Authorizing `identity:get_user(user_id=94bb3d71f58b48b684498cff082f5ce0)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:32.367207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c1efa-d5cf-43ae-96a0-32421e1da567 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:32.368734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 567/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:32 2022] GET /identity/v3/users/94bb3d71f58b48b684498cff082f5ce0 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:32.374951 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60b230-b675-40ae-98da-f5128647cdf0 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:32.390417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:32.472093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60b230-b675-40ae-98da-f5128647cdf0 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:32.503487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc02e7-e8fc-40d3-b643-0bb05c67b823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:32.504074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc02e7-e8fc-40d3-b643-0bb05c67b823 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:32.504348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc02e7-e8fc-40d3-b643-0bb05c67b823 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:32.504617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc02e7-e8fc-40d3-b643-0bb05c67b823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:32.568795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc02e7-e8fc-40d3-b643-0bb05c67b823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:32.570251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc02e7-e8fc-40d3-b643-0bb05c67b823 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:32.584928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:32.664835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 561/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:10:32.670606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c1efa-d5cf-43ae-96a0-32421e1da567 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:32.761459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:32.765951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c1efa-d5cf-43ae-96a0-32421e1da567 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:32.797672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b0530-4fad-48a0-8a3a-bed0d199be08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:32.798279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b0530-4fad-48a0-8a3a-bed0d199be08 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:32.798554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b0530-4fad-48a0-8a3a-bed0d199be08 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:32.798820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b0530-4fad-48a0-8a3a-bed0d199be08 None admin] PATH_INFO: `/v3/projects/024302e671b94c1ab0275621f3fdb02b` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:32.799897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b0530-4fad-48a0-8a3a-bed0d199be08 None admin] RBAC: Authorizing `identity:get_project(project_id=024302e671b94c1ab0275621f3fdb02b)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:32.801286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b0530-4fad-48a0-8a3a-bed0d199be08 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:32.802879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 568/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:10:32 2022] GET /identity/v3/projects/024302e671b94c1ab0275621f3fdb02b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:32.808791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc02e7-e8fc-40d3-b643-0bb05c67b823 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:32.904126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc02e7-e8fc-40d3-b643-0bb05c67b823 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:32.917467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:32.937633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972b81c-7065-4061-9931-b8c9d9db2577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:32.938225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972b81c-7065-4061-9931-b8c9d9db2577 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:32.938503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972b81c-7065-4061-9931-b8c9d9db2577 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:32.938774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972b81c-7065-4061-9931-b8c9d9db2577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:32.939732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972b81c-7065-4061-9931-b8c9d9db2577 None admin] RBAC: Adding query filter params (role.id=6a600852dc8442fdb2ffc3d663e274ff, scope.project.id=024302e671b94c1ab0275621f3fdb02b, user.id=94bb3d71f58b48b684498cff082f5ce0) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:32.940037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972b81c-7065-4061-9931-b8c9d9db2577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:32.941441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972b81c-7065-4061-9931-b8c9d9db2577 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:32.947029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 562/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:10:32 2022] GET /identity/v3/role_assignments?user.id=94bb3d71f58b48b684498cff082f5ce0&scope.project.id=024302e671b94c1ab0275621f3fdb02b&role.id=6a600852dc8442fdb2ffc3d663e274ff => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:35.065799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51237a76-66a7-48d6-ba76-6955e4171d7b None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:35.066343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51237a76-66a7-48d6-ba76-6955e4171d7b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:35.066343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51237a76-66a7-48d6-ba76-6955e4171d7b None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:35.067816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 569/1131] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:35.073140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9bd40-c51f-4fda-a73a-87442106f903 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:35.073562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9bd40-c51f-4fda-a73a-87442106f903 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:35.073709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9bd40-c51f-4fda-a73a-87442106f903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:35.091878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-87c9bd40-c51f-4fda-a73a-87442106f903 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:35.093055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-87c9bd40-c51f-4fda-a73a-87442106f903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:35.156034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 563/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:35.434539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51237a76-66a7-48d6-ba76-6955e4171d7b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:35.534317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51237a76-66a7-48d6-ba76-6955e4171d7b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:35.568542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253f94a-a375-426e-ac18-b6c337d54c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:35.569156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253f94a-a375-426e-ac18-b6c337d54c6c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:35.569428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253f94a-a375-426e-ac18-b6c337d54c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:35.569726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253f94a-a375-426e-ac18-b6c337d54c6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:35.570806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253f94a-a375-426e-ac18-b6c337d54c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:35.572215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253f94a-a375-426e-ac18-b6c337d54c6c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:35.573803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 570/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:35.580225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9bd40-c51f-4fda-a73a-87442106f903 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:35.675243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9bd40-c51f-4fda-a73a-87442106f903 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:35.684007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:35.709772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812cb03-1a86-4330-90b2-cf421548565c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:35.710299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812cb03-1a86-4330-90b2-cf421548565c None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:35.710572 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812cb03-1a86-4330-90b2-cf421548565c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:35.710838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812cb03-1a86-4330-90b2-cf421548565c None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:35.711486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812cb03-1a86-4330-90b2-cf421548565c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:35.712900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812cb03-1a86-4330-90b2-cf421548565c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:35.739504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 564/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:10:35 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:10:37.873766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8c164-c169-4924-9646-56b2e46f15b0 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:37.874315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8c164-c169-4924-9646-56b2e46f15b0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:37.874315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8c164-c169-4924-9646-56b2e46f15b0 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:37.875746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 571/1135] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:37.880907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c69dc-2e1c-4701-a8d3-5585118ec4c7 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:37.881315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c69dc-2e1c-4701-a8d3-5585118ec4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:37.881450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c69dc-2e1c-4701-a8d3-5585118ec4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:37.899511 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b59c69dc-2e1c-4701-a8d3-5585118ec4c7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:37.900675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b59c69dc-2e1c-4701-a8d3-5585118ec4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:37.964036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 565/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:38.230365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8c164-c169-4924-9646-56b2e46f15b0 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:38.264871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:38.329367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8c164-c169-4924-9646-56b2e46f15b0 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:38.361293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b20bb-4ef7-4558-aac4-95a170af0830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:38.361921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b20bb-4ef7-4558-aac4-95a170af0830 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:38.362196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b20bb-4ef7-4558-aac4-95a170af0830 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:38.362463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b20bb-4ef7-4558-aac4-95a170af0830 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:38.363536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b20bb-4ef7-4558-aac4-95a170af0830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:38.364943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b20bb-4ef7-4558-aac4-95a170af0830 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:38.366477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 572/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:10:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:38.372045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c69dc-2e1c-4701-a8d3-5585118ec4c7 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:38.457550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:38.467876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c69dc-2e1c-4701-a8d3-5585118ec4c7 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:38.499635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435b5f9-6fe2-4792-a4d1-b5ab7985c21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:38.500165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435b5f9-6fe2-4792-a4d1-b5ab7985c21b None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:38.500435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435b5f9-6fe2-4792-a4d1-b5ab7985c21b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:38.500699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435b5f9-6fe2-4792-a4d1-b5ab7985c21b None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:38.501327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435b5f9-6fe2-4792-a4d1-b5ab7985c21b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:38.502759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435b5f9-6fe2-4792-a4d1-b5ab7985c21b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:38.522888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 566/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:10:38 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:10:40.678111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db854cc8-af22-4c65-80b4-0553b1de2c13 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:40.678667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db854cc8-af22-4c65-80b4-0553b1de2c13 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:40.678667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db854cc8-af22-4c65-80b4-0553b1de2c13 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:40.679982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 573/1139] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:40.685253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f2561-d8f1-410f-9e14-adcb81aa983d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:40.685689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f2561-d8f1-410f-9e14-adcb81aa983d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:40.685824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f2561-d8f1-410f-9e14-adcb81aa983d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:40.704932 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7c9f2561-d8f1-410f-9e14-adcb81aa983d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:40.706138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7c9f2561-d8f1-410f-9e14-adcb81aa983d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:40.769568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 567/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:10:41.037052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db854cc8-af22-4c65-80b4-0553b1de2c13 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:41.134296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db854cc8-af22-4c65-80b4-0553b1de2c13 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:41.135835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:41.168864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039eee5-9e82-4585-a9a6-3a8718351bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:41.169467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039eee5-9e82-4585-a9a6-3a8718351bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:41.169769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039eee5-9e82-4585-a9a6-3a8718351bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:41.170033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039eee5-9e82-4585-a9a6-3a8718351bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:41.179724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039eee5-9e82-4585-a9a6-3a8718351bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-6039eee5-9e82-4585-a9a6-3a8718351bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:10:41.184325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:41.185483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:41.187081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 574/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:10:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:41.192278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f2561-d8f1-410f-9e14-adcb81aa983d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:41.265985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:41.288420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f2561-d8f1-410f-9e14-adcb81aa983d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:41.322476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1ccf6-b8d2-49be-8a51-b0f34dc402cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:41.323066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1ccf6-b8d2-49be-8a51-b0f34dc402cf None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:41.323341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1ccf6-b8d2-49be-8a51-b0f34dc402cf None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:41.323609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1ccf6-b8d2-49be-8a51-b0f34dc402cf None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:41.324259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1ccf6-b8d2-49be-8a51-b0f34dc402cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:41.324556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1ccf6-b8d2-49be-8a51-b0f34dc402cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:41.325939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1ccf6-b8d2-49be-8a51-b0f34dc402cf None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:41.333727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 568/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:10:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:41.339617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039eee5-9e82-4585-a9a6-3a8718351bb1 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:41.434056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039eee5-9e82-4585-a9a6-3a8718351bb1 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:41.465955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1d911-c344-4841-9a4e-2980c8f44295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:41.466550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1d911-c344-4841-9a4e-2980c8f44295 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:41.466822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1d911-c344-4841-9a4e-2980c8f44295 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:41.467104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1d911-c344-4841-9a4e-2980c8f44295 None admin] PATH_INFO: `/v3/users/8008a74a07fe419097e09c92d7d1211e` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:41.479465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1d911-c344-4841-9a4e-2980c8f44295 None admin] RBAC: Authorizing `identity:get_user(user_id=8008a74a07fe419097e09c92d7d1211e)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:41.480875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1d911-c344-4841-9a4e-2980c8f44295 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:41.482431 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 575/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:41 2022] GET /identity/v3/users/8008a74a07fe419097e09c92d7d1211e => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:41.488459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1ccf6-b8d2-49be-8a51-b0f34dc402cf None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:41.584422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1ccf6-b8d2-49be-8a51-b0f34dc402cf None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:41.616099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf61fc-4ff2-4a38-87fd-15f94abefe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:41.616685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf61fc-4ff2-4a38-87fd-15f94abefe02 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:41.616959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf61fc-4ff2-4a38-87fd-15f94abefe02 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:41.617228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf61fc-4ff2-4a38-87fd-15f94abefe02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:41.625853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:41.681813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf61fc-4ff2-4a38-87fd-15f94abefe02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:41.683231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf61fc-4ff2-4a38-87fd-15f94abefe02 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:41.775271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 569/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:10:41.781661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1d911-c344-4841-9a4e-2980c8f44295 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:41.803253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:41.877757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1d911-c344-4841-9a4e-2980c8f44295 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:41.909989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba0a36-dc67-4230-b084-843e0cc496f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:41.910598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba0a36-dc67-4230-b084-843e0cc496f5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:41.910869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba0a36-dc67-4230-b084-843e0cc496f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:41.911133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba0a36-dc67-4230-b084-843e0cc496f5 None admin] PATH_INFO: `/v3/projects/ddc4a404398f4a63b11d09896470f310` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:41.912215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba0a36-dc67-4230-b084-843e0cc496f5 None admin] RBAC: Authorizing `identity:get_project(project_id=ddc4a404398f4a63b11d09896470f310)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:41.913645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba0a36-dc67-4230-b084-843e0cc496f5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:41.915227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 576/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:10:41 2022] GET /identity/v3/projects/ddc4a404398f4a63b11d09896470f310 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:41.921965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf61fc-4ff2-4a38-87fd-15f94abefe02 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:41.959327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:42.017155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf61fc-4ff2-4a38-87fd-15f94abefe02 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:42.049774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da9d86-03b4-4742-af01-3ff26a8be896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:42.050374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da9d86-03b4-4742-af01-3ff26a8be896 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:42.050651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da9d86-03b4-4742-af01-3ff26a8be896 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:42.050923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da9d86-03b4-4742-af01-3ff26a8be896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:42.051638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da9d86-03b4-4742-af01-3ff26a8be896 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=ddc4a404398f4a63b11d09896470f310, user.id=8008a74a07fe419097e09c92d7d1211e) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:42.051944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da9d86-03b4-4742-af01-3ff26a8be896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:42.053358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da9d86-03b4-4742-af01-3ff26a8be896 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:42.058528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 570/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:10:41 2022] GET /identity/v3/role_assignments?user.id=8008a74a07fe419097e09c92d7d1211e&scope.project.id=ddc4a404398f4a63b11d09896470f310&role.id=36331ec562ff42f883fc883363bc4c8d => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:44.171606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084078f-b635-4924-9972-117dea470cf2 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:44.172099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084078f-b635-4924-9972-117dea470cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:44.172099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084078f-b635-4924-9972-117dea470cf2 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:44.173362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 577/1147] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:44.178633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6c0d9-493a-4815-8291-c973a604da17 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:44.179046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6c0d9-493a-4815-8291-c973a604da17 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:44.179172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6c0d9-493a-4815-8291-c973a604da17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:44.197777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-38d6c0d9-493a-4815-8291-c973a604da17 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:44.198940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-38d6c0d9-493a-4815-8291-c973a604da17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:44.261872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 571/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:10:44.525335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084078f-b635-4924-9972-117dea470cf2 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:44.610228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:44.622602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084078f-b635-4924-9972-117dea470cf2 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:44.654712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf4430-d8f6-4984-a29f-9eacc4d73b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:44.655316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf4430-d8f6-4984-a29f-9eacc4d73b08 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:44.655588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf4430-d8f6-4984-a29f-9eacc4d73b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:44.655854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf4430-d8f6-4984-a29f-9eacc4d73b08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:44.665585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf4430-d8f6-4984-a29f-9eacc4d73b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-7cbf4430-d8f6-4984-a29f-9eacc4d73b08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:10:44.670256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:44.671300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:44.672393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 578/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:10:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:44.677203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6c0d9-493a-4815-8291-c973a604da17 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:44.707419 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:44.772832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6c0d9-493a-4815-8291-c973a604da17 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:44.805845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446466ee-1227-4fba-873c-18cb51fbf412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:44.806377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446466ee-1227-4fba-873c-18cb51fbf412 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:44.806652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446466ee-1227-4fba-873c-18cb51fbf412 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:44.806919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446466ee-1227-4fba-873c-18cb51fbf412 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:44.807568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446466ee-1227-4fba-873c-18cb51fbf412 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:44.807864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446466ee-1227-4fba-873c-18cb51fbf412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:44.809215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446466ee-1227-4fba-873c-18cb51fbf412 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:44.817248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 572/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:10:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:44.822939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf4430-d8f6-4984-a29f-9eacc4d73b08 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:44.918293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf4430-d8f6-4984-a29f-9eacc4d73b08 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:44.951285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1d60a-a129-4a31-b6ec-1c99f722e080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:44.951924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1d60a-a129-4a31-b6ec-1c99f722e080 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:44.952201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1d60a-a129-4a31-b6ec-1c99f722e080 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:44.952478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1d60a-a129-4a31-b6ec-1c99f722e080 None admin] PATH_INFO: `/v3/users/8008a74a07fe419097e09c92d7d1211e` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:44.953600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1d60a-a129-4a31-b6ec-1c99f722e080 None admin] RBAC: Authorizing `identity:get_user(user_id=8008a74a07fe419097e09c92d7d1211e)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:44.955017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b1d60a-a129-4a31-b6ec-1c99f722e080 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:44.956597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 579/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:44 2022] GET /identity/v3/users/8008a74a07fe419097e09c92d7d1211e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:44.962382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446466ee-1227-4fba-873c-18cb51fbf412 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:45.054538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:45.061863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446466ee-1227-4fba-873c-18cb51fbf412 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:45.099122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff36b7-3097-4f2b-b4ab-2db816fe17a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:45.099738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff36b7-3097-4f2b-b4ab-2db816fe17a7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:45.100135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff36b7-3097-4f2b-b4ab-2db816fe17a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:45.100569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff36b7-3097-4f2b-b4ab-2db816fe17a7 None admin] PATH_INFO: `/v3/projects/ddc4a404398f4a63b11d09896470f310` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:45.101699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff36b7-3097-4f2b-b4ab-2db816fe17a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ddc4a404398f4a63b11d09896470f310)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:45.103200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff36b7-3097-4f2b-b4ab-2db816fe17a7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:45.104819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 573/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:10:44 2022] GET /identity/v3/projects/ddc4a404398f4a63b11d09896470f310 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:45.111414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1d60a-a129-4a31-b6ec-1c99f722e080 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:45.129764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:45.208664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1d60a-a129-4a31-b6ec-1c99f722e080 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:45.241633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38ea10-c639-4789-8adb-d792ffbd3c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:45.242273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38ea10-c639-4789-8adb-d792ffbd3c5c None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:45.242549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38ea10-c639-4789-8adb-d792ffbd3c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:45.242818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38ea10-c639-4789-8adb-d792ffbd3c5c None admin] PATH_INFO: `/v3/projects/ddc4a404398f4a63b11d09896470f310/users/8008a74a07fe419097e09c92d7d1211e/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:45.244665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38ea10-c639-4789-8adb-d792ffbd3c5c None admin] RBAC: Authorizing `identity:create_grant(project_id=ddc4a404398f4a63b11d09896470f310, user_id=8008a74a07fe419097e09c92d7d1211e, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:45.246108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38ea10-c639-4789-8adb-d792ffbd3c5c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:45.260023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 580/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:10:45 2022] PUT /identity/v3/projects/ddc4a404398f4a63b11d09896470f310/users/8008a74a07fe419097e09c92d7d1211e/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:10:47.397446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789fb12-db3a-4f64-a833-f8894778f2a7 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:47.398048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789fb12-db3a-4f64-a833-f8894778f2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:47.398048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789fb12-db3a-4f64-a833-f8894778f2a7 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:47.399349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 574/1154] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:47.405241 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875465f-8bba-46dd-a46d-5b4945af4b87 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:47.405703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875465f-8bba-46dd-a46d-5b4945af4b87 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:47.405810 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875465f-8bba-46dd-a46d-5b4945af4b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:47.423816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a875465f-8bba-46dd-a46d-5b4945af4b87 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:47.424982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a875465f-8bba-46dd-a46d-5b4945af4b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:47.489616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 581/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:47.755064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789fb12-db3a-4f64-a833-f8894778f2a7 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:47.852425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789fb12-db3a-4f64-a833-f8894778f2a7 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:47.886025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b27ee3-817f-4cba-ba9d-4b02b446d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:47.886644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b27ee3-817f-4cba-ba9d-4b02b446d83c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:47.886921 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b27ee3-817f-4cba-ba9d-4b02b446d83c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:47.887192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b27ee3-817f-4cba-ba9d-4b02b446d83c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:47.896910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b27ee3-817f-4cba-ba9d-4b02b446d83c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-d9b27ee3-817f-4cba-ba9d-4b02b446d83c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:10:47.901710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:47.902877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:10:47.903912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 575/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:10:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:47.909351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875465f-8bba-46dd-a46d-5b4945af4b87 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:47.917256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:48.005476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875465f-8bba-46dd-a46d-5b4945af4b87 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:48.038509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911e92d-6023-4c27-bed2-5903babef3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:48.039098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911e92d-6023-4c27-bed2-5903babef3f8 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:48.039373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911e92d-6023-4c27-bed2-5903babef3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:48.039636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911e92d-6023-4c27-bed2-5903babef3f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:48.040280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911e92d-6023-4c27-bed2-5903babef3f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:48.040575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911e92d-6023-4c27-bed2-5903babef3f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:48.041979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911e92d-6023-4c27-bed2-5903babef3f8 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:48.049737 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 582/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:10:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:48.055519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b27ee3-817f-4cba-ba9d-4b02b446d83c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:48.080063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:48.151904 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b27ee3-817f-4cba-ba9d-4b02b446d83c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:48.184251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3326963-4ca4-4fcc-821b-39968860e0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:48.184857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3326963-4ca4-4fcc-821b-39968860e0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:48.185129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3326963-4ca4-4fcc-821b-39968860e0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:48.185395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3326963-4ca4-4fcc-821b-39968860e0e7 None admin] PATH_INFO: `/v3/users/8008a74a07fe419097e09c92d7d1211e` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:48.186513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3326963-4ca4-4fcc-821b-39968860e0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=8008a74a07fe419097e09c92d7d1211e)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:48.187922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3326963-4ca4-4fcc-821b-39968860e0e7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:48.189457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 576/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:10:48 2022] GET /identity/v3/users/8008a74a07fe419097e09c92d7d1211e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:48.195318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911e92d-6023-4c27-bed2-5903babef3f8 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:48.258416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:48.291134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911e92d-6023-4c27-bed2-5903babef3f8 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:48.322611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1637f24-e965-4bcc-85a3-54f93fe20e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:48.323195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1637f24-e965-4bcc-85a3-54f93fe20e8e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:48.323468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1637f24-e965-4bcc-85a3-54f93fe20e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:48.323732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1637f24-e965-4bcc-85a3-54f93fe20e8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:48.386878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1637f24-e965-4bcc-85a3-54f93fe20e8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:48.388287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1637f24-e965-4bcc-85a3-54f93fe20e8e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:48.450168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:48.483058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 583/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:10:48.488888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3326963-4ca4-4fcc-821b-39968860e0e7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:48.582154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:48.591747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3326963-4ca4-4fcc-821b-39968860e0e7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:48.623095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc2c56-ba20-4434-8439-0f8c5840af88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:48.624224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc2c56-ba20-4434-8439-0f8c5840af88 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:48.624498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc2c56-ba20-4434-8439-0f8c5840af88 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:48.624761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc2c56-ba20-4434-8439-0f8c5840af88 None admin] PATH_INFO: `/v3/projects/ddc4a404398f4a63b11d09896470f310` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:48.625873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc2c56-ba20-4434-8439-0f8c5840af88 None admin] RBAC: Authorizing `identity:get_project(project_id=ddc4a404398f4a63b11d09896470f310)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:48.627288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc2c56-ba20-4434-8439-0f8c5840af88 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:48.628890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 577/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:10:48 2022] GET /identity/v3/projects/ddc4a404398f4a63b11d09896470f310 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:48.634934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1637f24-e965-4bcc-85a3-54f93fe20e8e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:48.730097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1637f24-e965-4bcc-85a3-54f93fe20e8e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:48.762304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e52766-9a21-468d-947b-74339232f58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:48.762894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52766-9a21-468d-947b-74339232f58d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:48.763168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52766-9a21-468d-947b-74339232f58d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:48.763432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52766-9a21-468d-947b-74339232f58d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:48.764696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52766-9a21-468d-947b-74339232f58d None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=ddc4a404398f4a63b11d09896470f310, user.id=8008a74a07fe419097e09c92d7d1211e) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:48.764994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52766-9a21-468d-947b-74339232f58d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:48.766413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52766-9a21-468d-947b-74339232f58d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:48.771861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 584/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:10:48 2022] GET /identity/v3/role_assignments?user.id=8008a74a07fe419097e09c92d7d1211e&scope.project.id=ddc4a404398f4a63b11d09896470f310&role.id=36331ec562ff42f883fc883363bc4c8d => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:50.913862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cdb2a-7822-4abd-81ed-9b606b53d7c4 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:50.914289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cdb2a-7822-4abd-81ed-9b606b53d7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:50.914399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cdb2a-7822-4abd-81ed-9b606b53d7c4 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:50.915724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 578/1162] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:50.921687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207ae55-2251-493a-a7bd-9c5b25f3e5a6 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:50.922099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207ae55-2251-493a-a7bd-9c5b25f3e5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:50.922226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207ae55-2251-493a-a7bd-9c5b25f3e5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:50.940756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-b207ae55-2251-493a-a7bd-9c5b25f3e5a6 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:50.941954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b207ae55-2251-493a-a7bd-9c5b25f3e5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:50.966414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:51.006702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 585/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:10:51.273877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cdb2a-7822-4abd-81ed-9b606b53d7c4 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:51.370029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cdb2a-7822-4abd-81ed-9b606b53d7c4 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:51.402981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40eba7-fc49-4e11-af59-68ae7b15867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:51.403568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40eba7-fc49-4e11-af59-68ae7b15867c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:51.403840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40eba7-fc49-4e11-af59-68ae7b15867c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:51.404105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40eba7-fc49-4e11-af59-68ae7b15867c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:51.429476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:51.469912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40eba7-fc49-4e11-af59-68ae7b15867c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:51.471332 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40eba7-fc49-4e11-af59-68ae7b15867c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:51.563476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 579/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:10:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:10:51.569599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207ae55-2251-493a-a7bd-9c5b25f3e5a6 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:51.663350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207ae55-2251-493a-a7bd-9c5b25f3e5a6 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:51.695776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d263dbd-b7a7-408e-b92a-e9751996c02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:51.696391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d263dbd-b7a7-408e-b92a-e9751996c02e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:51.696667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d263dbd-b7a7-408e-b92a-e9751996c02e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:51.696935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d263dbd-b7a7-408e-b92a-e9751996c02e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:51.703949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d263dbd-b7a7-408e-b92a-e9751996c02e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:51.705365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d263dbd-b7a7-408e-b92a-e9751996c02e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-9d263dbd-b7a7-408e-b92a-e9751996c02e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:10:51.716314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:10:51.717299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:10:51.718237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 586/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:10:51 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:10:51.723195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40eba7-fc49-4e11-af59-68ae7b15867c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:51.778175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:51.819182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40eba7-fc49-4e11-af59-68ae7b15867c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:51.850743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d2b7a-2f09-4bdc-83ab-949e193fcff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:51.851339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d2b7a-2f09-4bdc-83ab-949e193fcff4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:51.851618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d2b7a-2f09-4bdc-83ab-949e193fcff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:51.851890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d2b7a-2f09-4bdc-83ab-949e193fcff4 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:51.852523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d2b7a-2f09-4bdc-83ab-949e193fcff4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:10:51.852825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d2b7a-2f09-4bdc-83ab-949e193fcff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:51.854228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d2b7a-2f09-4bdc-83ab-949e193fcff4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:51.861601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 580/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 20:10:51 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:53.981947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49902aa-bfc8-4ccd-8dc7-7c8cb9751da0 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:53.982448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49902aa-bfc8-4ccd-8dc7-7c8cb9751da0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:53.982566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49902aa-bfc8-4ccd-8dc7-7c8cb9751da0 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:53.983858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 587/1167] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:53.989014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a24bd5-de40-4a95-b847-ff3a10c1d203 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:53.989397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a24bd5-de40-4a95-b847-ff3a10c1d203 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:53.989574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a24bd5-de40-4a95-b847-ff3a10c1d203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:54.007618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-02a24bd5-de40-4a95-b847-ff3a10c1d203 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:54.008781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-02a24bd5-de40-4a95-b847-ff3a10c1d203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:54.072487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 581/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:54.337499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49902aa-bfc8-4ccd-8dc7-7c8cb9751da0 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:54.361985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:54.436141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49902aa-bfc8-4ccd-8dc7-7c8cb9751da0 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:54.467971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1974e7-8dd0-40a4-a18b-a7bf599b913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:54.468554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1974e7-8dd0-40a4-a18b-a7bf599b913b None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:54.468826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1974e7-8dd0-40a4-a18b-a7bf599b913b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:54.469093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1974e7-8dd0-40a4-a18b-a7bf599b913b None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:54.469766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1974e7-8dd0-40a4-a18b-a7bf599b913b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:54.471107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1974e7-8dd0-40a4-a18b-a7bf599b913b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:54.486387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 588/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Wed Oct 5 20:10:54 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:10:56.611252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247d1ba-cc83-4621-8d5f-579b2f454cbb None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:56.611809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247d1ba-cc83-4621-8d5f-579b2f454cbb None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:56.611809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247d1ba-cc83-4621-8d5f-579b2f454cbb None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:56.613124 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 582/1170] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:56.618420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc459-a83a-47ce-8fbe-b3b742f11b1e None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:56.618854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc459-a83a-47ce-8fbe-b3b742f11b1e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:56.618960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc459-a83a-47ce-8fbe-b3b742f11b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:56.637353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-98fdc459-a83a-47ce-8fbe-b3b742f11b1e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:56.638620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-98fdc459-a83a-47ce-8fbe-b3b742f11b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:56.702532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 589/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:56.965964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247d1ba-cc83-4621-8d5f-579b2f454cbb None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:57.018623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:57.066091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247d1ba-cc83-4621-8d5f-579b2f454cbb None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:57.098562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ce635-2a39-4f6e-8b7d-a4954deb9e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:57.099173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ce635-2a39-4f6e-8b7d-a4954deb9e62 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:57.099445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ce635-2a39-4f6e-8b7d-a4954deb9e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:57.099711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ce635-2a39-4f6e-8b7d-a4954deb9e62 None admin] PATH_INFO: `/v3/domains/f9001dc083ef4e4b9fb4f7c2ead6c620` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:57.110222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ce635-2a39-4f6e-8b7d-a4954deb9e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f9001dc083ef4e4b9fb4f7c2ead6c620)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:57.111656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629ce635-2a39-4f6e-8b7d-a4954deb9e62 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:57.113208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 583/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Wed Oct 5 20:10:56 2022] GET /identity/v3/domains/f9001dc083ef4e4b9fb4f7c2ead6c620 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:57.119005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdc459-a83a-47ce-8fbe-b3b742f11b1e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:57.133340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:57.215161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdc459-a83a-47ce-8fbe-b3b742f11b1e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:57.247249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a99b1a-042b-4eec-8310-980a30a280c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:57.248423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99b1a-042b-4eec-8310-980a30a280c5 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:57.248710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99b1a-042b-4eec-8310-980a30a280c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:57.248978 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99b1a-042b-4eec-8310-980a30a280c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:57.249635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a99b1a-042b-4eec-8310-980a30a280c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:57.251044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a99b1a-042b-4eec-8310-980a30a280c5 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:57.276665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 590/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:10:57 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:10:59.426973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f0695-6959-41ce-9ba4-bfe9c3dcce66 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:59.427514 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f0695-6959-41ce-9ba4-bfe9c3dcce66 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:59.427514 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f0695-6959-41ce-9ba4-bfe9c3dcce66 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:59.428843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 584/1174] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:10:59.434137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e2553-159b-4412-be14-4dc67ced9352 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:59.434546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e2553-159b-4412-be14-4dc67ced9352 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:59.434682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e2553-159b-4412-be14-4dc67ced9352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:59.453141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-fc7e2553-159b-4412-be14-4dc67ced9352 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:10:59.454331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fc7e2553-159b-4412-be14-4dc67ced9352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:10:59.511961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:59.521251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 591/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:10:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:10:59.785501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f0695-6959-41ce-9ba4-bfe9c3dcce66 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:10:59.883502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f0695-6959-41ce-9ba4-bfe9c3dcce66 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:10:59.896247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:10:59.917141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc804ce-3e81-4f7f-a8ab-a93ca4334047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:10:59.917794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc804ce-3e81-4f7f-a8ab-a93ca4334047 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:10:59.918073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc804ce-3e81-4f7f-a8ab-a93ca4334047 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:10:59.918346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc804ce-3e81-4f7f-a8ab-a93ca4334047 None admin] PATH_INFO: `/v3/domains/f9001dc083ef4e4b9fb4f7c2ead6c620` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:10:59.919450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc804ce-3e81-4f7f-a8ab-a93ca4334047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f9001dc083ef4e4b9fb4f7c2ead6c620)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:10:59.921539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc804ce-3e81-4f7f-a8ab-a93ca4334047 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:10:59.923085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 585/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Wed Oct 5 20:10:59 2022] GET /identity/v3/domains/f9001dc083ef4e4b9fb4f7c2ead6c620 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:10:59.928907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e2553-159b-4412-be14-4dc67ced9352 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:00.022426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e2553-159b-4412-be14-4dc67ced9352 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:00.054398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18fee5-ce0c-4da9-bcc3-02067777e316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:00.054927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18fee5-ce0c-4da9-bcc3-02067777e316 None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:00.055200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18fee5-ce0c-4da9-bcc3-02067777e316 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:00.055465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18fee5-ce0c-4da9-bcc3-02067777e316 None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:00.056104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18fee5-ce0c-4da9-bcc3-02067777e316 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:00.057500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18fee5-ce0c-4da9-bcc3-02067777e316 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:00.077843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 592/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:10:59 2022] POST /identity/v3/users => generated 311 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:11:02.237161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c2707-34b1-4bfb-b0af-9f8ed1ed6040 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:02.237704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c2707-34b1-4bfb-b0af-9f8ed1ed6040 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:02.237704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c2707-34b1-4bfb-b0af-9f8ed1ed6040 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:02.238995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 586/1178] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:11:02.244219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59d6f0-04a3-43e4-a459-be8b18af1e5e None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:02.244634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59d6f0-04a3-43e4-a459-be8b18af1e5e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:02.244754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59d6f0-04a3-43e4-a459-be8b18af1e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:02.263030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-dd59d6f0-04a3-43e4-a459-be8b18af1e5e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:11:02.264792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-dd59d6f0-04a3-43e4-a459-be8b18af1e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:11:02.308616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:02.330073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 593/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:11:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:11:02.598046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c2707-34b1-4bfb-b0af-9f8ed1ed6040 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:02.694772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c2707-34b1-4bfb-b0af-9f8ed1ed6040 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:02.726648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5496d-74bf-4a24-9b98-64ba93bf03c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:02.727246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5496d-74bf-4a24-9b98-64ba93bf03c5 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:02.727516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5496d-74bf-4a24-9b98-64ba93bf03c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:02.727780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5496d-74bf-4a24-9b98-64ba93bf03c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:02.737845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5496d-74bf-4a24-9b98-64ba93bf03c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-81e5496d-74bf-4a24-9b98-64ba93bf03c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:11:02.742466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:11:02.743530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:11:02.744706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 587/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:11:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:11:02.749417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59d6f0-04a3-43e4-a459-be8b18af1e5e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:02.851713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59d6f0-04a3-43e4-a459-be8b18af1e5e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:02.885048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ddebf8-f55a-43ba-980a-ed5a79822ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:02.885682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddebf8-f55a-43ba-980a-ed5a79822ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:02.885962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddebf8-f55a-43ba-980a-ed5a79822ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:02.886231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ddebf8-f55a-43ba-980a-ed5a79822ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:02.886888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ddebf8-f55a-43ba-980a-ed5a79822ab3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:11:02.887189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ddebf8-f55a-43ba-980a-ed5a79822ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:02.888552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ddebf8-f55a-43ba-980a-ed5a79822ab3 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:02.896364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 594/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:11:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:11:02.902259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5496d-74bf-4a24-9b98-64ba93bf03c5 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:02.943303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:02.998513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5496d-74bf-4a24-9b98-64ba93bf03c5 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:03.030861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed950871-7c99-4cfc-9f1b-0c00e1ed1c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:03.031402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed950871-7c99-4cfc-9f1b-0c00e1ed1c22 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:03.031677 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed950871-7c99-4cfc-9f1b-0c00e1ed1c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:03.031946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed950871-7c99-4cfc-9f1b-0c00e1ed1c22 None admin] PATH_INFO: `/v3/users/b77194d1db8747d1801f389aa6fc58e4` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:03.044139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed950871-7c99-4cfc-9f1b-0c00e1ed1c22 None admin] RBAC: Authorizing `identity:get_user(user_id=b77194d1db8747d1801f389aa6fc58e4)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:03.045627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed950871-7c99-4cfc-9f1b-0c00e1ed1c22 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:03.047176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 588/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:11:02 2022] GET /identity/v3/users/b77194d1db8747d1801f389aa6fc58e4 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:11:03.052945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ddebf8-f55a-43ba-980a-ed5a79822ab3 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:03.096085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:03.151796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ddebf8-f55a-43ba-980a-ed5a79822ab3 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:03.184415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7e3f3-8c2c-4f76-9bd2-7a259336747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:03.185012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7e3f3-8c2c-4f76-9bd2-7a259336747d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:03.185284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7e3f3-8c2c-4f76-9bd2-7a259336747d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:03.185577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7e3f3-8c2c-4f76-9bd2-7a259336747d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:03.249155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7e3f3-8c2c-4f76-9bd2-7a259336747d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:03.250616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7e3f3-8c2c-4f76-9bd2-7a259336747d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:03.290726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:03.346273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 595/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:11:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:11:03.352342 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed950871-7c99-4cfc-9f1b-0c00e1ed1c22 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:03.445360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:03.449317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed950871-7c99-4cfc-9f1b-0c00e1ed1c22 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:03.483067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd146243-c62b-4fe0-82bf-0cc6228e8c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:03.483686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd146243-c62b-4fe0-82bf-0cc6228e8c95 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:03.483964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd146243-c62b-4fe0-82bf-0cc6228e8c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:03.484232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd146243-c62b-4fe0-82bf-0cc6228e8c95 None admin] PATH_INFO: `/v3/projects/9e8cbf6ffe6b4cf2a9765c4fce76876c` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:03.485328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd146243-c62b-4fe0-82bf-0cc6228e8c95 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8cbf6ffe6b4cf2a9765c4fce76876c)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:03.486782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd146243-c62b-4fe0-82bf-0cc6228e8c95 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:03.488386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 589/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:11:03 2022] GET /identity/v3/projects/9e8cbf6ffe6b4cf2a9765c4fce76876c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:11:03.494718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7e3f3-8c2c-4f76-9bd2-7a259336747d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:03.590445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7e3f3-8c2c-4f76-9bd2-7a259336747d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:03.623784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793875c-882f-4538-9c99-7927359de177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:03.624370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793875c-882f-4538-9c99-7927359de177 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:03.624646 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793875c-882f-4538-9c99-7927359de177 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:03.624916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793875c-882f-4538-9c99-7927359de177 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:03.625657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793875c-882f-4538-9c99-7927359de177 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=9e8cbf6ffe6b4cf2a9765c4fce76876c, user.id=b77194d1db8747d1801f389aa6fc58e4) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:11:03.625965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793875c-882f-4538-9c99-7927359de177 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:03.627372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793875c-882f-4538-9c99-7927359de177 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:03.632570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 596/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:11:03 2022] GET /identity/v3/role_assignments?user.id=b77194d1db8747d1801f389aa6fc58e4&scope.project.id=9e8cbf6ffe6b4cf2a9765c4fce76876c&role.id=36331ec562ff42f883fc883363bc4c8d => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:11:05.741273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fea971-ff08-43a8-957a-a273d2b41106 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:05.741858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fea971-ff08-43a8-957a-a273d2b41106 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:05.741858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fea971-ff08-43a8-957a-a273d2b41106 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:05.743163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 590/1186] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:11:05.748253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9f883-d449-441f-83a7-3bc525cb45a5 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:05.748666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9f883-d449-441f-83a7-3bc525cb45a5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:05.748813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9f883-d449-441f-83a7-3bc525cb45a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:05.761341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:05.766790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4ce9f883-d449-441f-83a7-3bc525cb45a5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:11:05.767947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4ce9f883-d449-441f-83a7-3bc525cb45a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:11:05.831965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 597/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:11:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:11:06.105249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fea971-ff08-43a8-957a-a273d2b41106 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:06.203508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fea971-ff08-43a8-957a-a273d2b41106 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:06.235764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9db2e-a540-425e-b4bc-90200316d632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:06.236372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9db2e-a540-425e-b4bc-90200316d632 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:06.236644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9db2e-a540-425e-b4bc-90200316d632 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:06.236910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9db2e-a540-425e-b4bc-90200316d632 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:06.246473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d9db2e-a540-425e-b4bc-90200316d632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-b9d9db2e-a540-425e-b4bc-90200316d632 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:11:06.251728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:11:06.252799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:11:06.253779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 591/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:11:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:11:06.258681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9f883-d449-441f-83a7-3bc525cb45a5 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:06.352576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9f883-d449-441f-83a7-3bc525cb45a5 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:06.385493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce1170-a34f-481e-afff-a2d4f40f665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:06.386070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce1170-a34f-481e-afff-a2d4f40f665c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:06.386343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce1170-a34f-481e-afff-a2d4f40f665c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:06.386607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce1170-a34f-481e-afff-a2d4f40f665c None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:06.387252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce1170-a34f-481e-afff-a2d4f40f665c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:11:06.387549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce1170-a34f-481e-afff-a2d4f40f665c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:06.388895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce1170-a34f-481e-afff-a2d4f40f665c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:06.390989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:06.397215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 598/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:11:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:11:06.402947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9db2e-a540-425e-b4bc-90200316d632 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:06.430497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:06.499382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9db2e-a540-425e-b4bc-90200316d632 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:06.531403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c8493-5c0e-4fa3-84f5-1a9d12d6446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:06.532000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c8493-5c0e-4fa3-84f5-1a9d12d6446f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:06.532271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c8493-5c0e-4fa3-84f5-1a9d12d6446f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:06.532535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c8493-5c0e-4fa3-84f5-1a9d12d6446f None admin] PATH_INFO: `/v3/users/b77194d1db8747d1801f389aa6fc58e4` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:06.533642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c8493-5c0e-4fa3-84f5-1a9d12d6446f None admin] RBAC: Authorizing `identity:get_user(user_id=b77194d1db8747d1801f389aa6fc58e4)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:06.535045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c8493-5c0e-4fa3-84f5-1a9d12d6446f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:06.536580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 592/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:11:06 2022] GET /identity/v3/users/b77194d1db8747d1801f389aa6fc58e4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:11:06.542334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce1170-a34f-481e-afff-a2d4f40f665c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:06.639786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce1170-a34f-481e-afff-a2d4f40f665c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:06.672462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8594a2-8900-445e-a2b2-fe34efdc4a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:06.673109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8594a2-8900-445e-a2b2-fe34efdc4a09 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:06.673387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8594a2-8900-445e-a2b2-fe34efdc4a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:06.673686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8594a2-8900-445e-a2b2-fe34efdc4a09 None admin] PATH_INFO: `/v3/projects/9e8cbf6ffe6b4cf2a9765c4fce76876c` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:06.674789 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8594a2-8900-445e-a2b2-fe34efdc4a09 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8cbf6ffe6b4cf2a9765c4fce76876c)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:06.676209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8594a2-8900-445e-a2b2-fe34efdc4a09 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:06.677852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 599/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:11:06 2022] GET /identity/v3/projects/9e8cbf6ffe6b4cf2a9765c4fce76876c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:11:06.683917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c8493-5c0e-4fa3-84f5-1a9d12d6446f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:06.773496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:06.781019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c8493-5c0e-4fa3-84f5-1a9d12d6446f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:06.812668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dbd0c-880d-4172-bf7e-c086797732c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:06.813305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dbd0c-880d-4172-bf7e-c086797732c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:06.813615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dbd0c-880d-4172-bf7e-c086797732c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:06.813888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dbd0c-880d-4172-bf7e-c086797732c2 None admin] PATH_INFO: `/v3/projects/9e8cbf6ffe6b4cf2a9765c4fce76876c/users/b77194d1db8747d1801f389aa6fc58e4/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:06.815976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dbd0c-880d-4172-bf7e-c086797732c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e8cbf6ffe6b4cf2a9765c4fce76876c, user_id=b77194d1db8747d1801f389aa6fc58e4, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:06.817385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dbd0c-880d-4172-bf7e-c086797732c2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:06.829924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 593/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:11:06 2022] PUT /identity/v3/projects/9e8cbf6ffe6b4cf2a9765c4fce76876c/users/b77194d1db8747d1801f389aa6fc58e4/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:11:08.943320 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b425a09-0887-403a-b2c8-cf7de1888ecf None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:08.943814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b425a09-0887-403a-b2c8-cf7de1888ecf None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:08.943904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b425a09-0887-403a-b2c8-cf7de1888ecf None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:08.945216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 600/1193] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:11:08.950589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996de70c-90bd-47fc-bf3e-944a54e65275 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:08.950976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996de70c-90bd-47fc-bf3e-944a54e65275 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:08.951133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996de70c-90bd-47fc-bf3e-944a54e65275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:08.969202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-996de70c-90bd-47fc-bf3e-944a54e65275 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:11:08.970403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-996de70c-90bd-47fc-bf3e-944a54e65275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:11:09.033971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 594/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:11:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:11:09.299979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b425a09-0887-403a-b2c8-cf7de1888ecf None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:09.364663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:09.399591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b425a09-0887-403a-b2c8-cf7de1888ecf None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:09.431825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bd57a-d326-4925-92c3-e14b8024abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:09.432439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bd57a-d326-4925-92c3-e14b8024abfb None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:09.432716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bd57a-d326-4925-92c3-e14b8024abfb None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:09.432986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bd57a-d326-4925-92c3-e14b8024abfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:09.444216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654bd57a-d326-4925-92c3-e14b8024abfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-654bd57a-d326-4925-92c3-e14b8024abfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:11:09.448955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:11:09.450040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:11:09.451781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 601/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:11:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:11:09.456488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996de70c-90bd-47fc-bf3e-944a54e65275 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:09.534652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:09.551800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996de70c-90bd-47fc-bf3e-944a54e65275 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:09.584466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb322ce-5522-4282-95f5-f0289067fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:09.585055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb322ce-5522-4282-95f5-f0289067fb04 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:09.585328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb322ce-5522-4282-95f5-f0289067fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:09.585620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb322ce-5522-4282-95f5-f0289067fb04 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:09.586265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb322ce-5522-4282-95f5-f0289067fb04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:11:09.586560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb322ce-5522-4282-95f5-f0289067fb04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:09.587902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb322ce-5522-4282-95f5-f0289067fb04 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:09.595513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 595/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:11:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:11:09.601397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bd57a-d326-4925-92c3-e14b8024abfb None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:09.696802 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bd57a-d326-4925-92c3-e14b8024abfb None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:09.720886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:09.729854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d35ca-e991-4dae-afad-4874f3d325b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:09.730463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d35ca-e991-4dae-afad-4874f3d325b4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:09.730736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d35ca-e991-4dae-afad-4874f3d325b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:09.731005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d35ca-e991-4dae-afad-4874f3d325b4 None admin] PATH_INFO: `/v3/users/b77194d1db8747d1801f389aa6fc58e4` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:09.732146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d35ca-e991-4dae-afad-4874f3d325b4 None admin] RBAC: Authorizing `identity:get_user(user_id=b77194d1db8747d1801f389aa6fc58e4)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:09.733578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d35ca-e991-4dae-afad-4874f3d325b4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:09.735114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 602/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 20:11:09 2022] GET /identity/v3/users/b77194d1db8747d1801f389aa6fc58e4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:11:09.740744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb322ce-5522-4282-95f5-f0289067fb04 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:09.837416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb322ce-5522-4282-95f5-f0289067fb04 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:09.869621 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e1953-a7ab-4a93-a8e5-68970d2626b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:09.870210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e1953-a7ab-4a93-a8e5-68970d2626b7 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:09.870486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e1953-a7ab-4a93-a8e5-68970d2626b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:09.870757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e1953-a7ab-4a93-a8e5-68970d2626b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:09.886253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:09.936981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e1953-a7ab-4a93-a8e5-68970d2626b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:09.938436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e1953-a7ab-4a93-a8e5-68970d2626b7 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:10.030549 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 596/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:11:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:11:10.036489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d35ca-e991-4dae-afad-4874f3d325b4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:10.132497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d35ca-e991-4dae-afad-4874f3d325b4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:10.164204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22e7da-74fd-4aed-868a-e460e7671413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:10.164807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22e7da-74fd-4aed-868a-e460e7671413 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:10.165083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22e7da-74fd-4aed-868a-e460e7671413 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:10.165352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22e7da-74fd-4aed-868a-e460e7671413 None admin] PATH_INFO: `/v3/projects/9e8cbf6ffe6b4cf2a9765c4fce76876c` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:10.166464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22e7da-74fd-4aed-868a-e460e7671413 None admin] RBAC: Authorizing `identity:get_project(project_id=9e8cbf6ffe6b4cf2a9765c4fce76876c)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:10.167872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f22e7da-74fd-4aed-868a-e460e7671413 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:10.170018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 603/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 20:11:10 2022] GET /identity/v3/projects/9e8cbf6ffe6b4cf2a9765c4fce76876c => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:11:10.175852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e1953-a7ab-4a93-a8e5-68970d2626b7 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:11:10.222331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:10.273525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e1953-a7ab-4a93-a8e5-68970d2626b7 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:11:10.305227 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bfab8-51ea-49fd-9e3c-3803a7239381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:11:10.305844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bfab8-51ea-49fd-9e3c-3803a7239381 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:11:10.306121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bfab8-51ea-49fd-9e3c-3803a7239381 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:11:10.306390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bfab8-51ea-49fd-9e3c-3803a7239381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:11:10.307096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bfab8-51ea-49fd-9e3c-3803a7239381 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=9e8cbf6ffe6b4cf2a9765c4fce76876c, user.id=b77194d1db8747d1801f389aa6fc58e4) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:11:10.307399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bfab8-51ea-49fd-9e3c-3803a7239381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:11:10.308811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bfab8-51ea-49fd-9e3c-3803a7239381 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:11:10.315088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 597/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:11:10 2022] GET /identity/v3/role_assignments?user.id=b77194d1db8747d1801f389aa6fc58e4&scope.project.id=9e8cbf6ffe6b4cf2a9765c4fce76876c&role.id=36331ec562ff42f883fc883363bc4c8d => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:11:20.160322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=69 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:11:20.312319 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=45 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:05.652292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692347-2cb1-4da9-b99b-e57b746b5f71 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:05.652948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692347-2cb1-4da9-b99b-e57b746b5f71 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:05.652948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4692347-2cb1-4da9-b99b-e57b746b5f71 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:05.654474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 604/1201] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:12:05.660465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989abf5a-1b39-4d50-a82b-552e651de059 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:05.661067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989abf5a-1b39-4d50-a82b-552e651de059 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:05.661067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989abf5a-1b39-4d50-a82b-552e651de059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:05.682069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-989abf5a-1b39-4d50-a82b-552e651de059 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:05.683307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-989abf5a-1b39-4d50-a82b-552e651de059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:05.769546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 598/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:12:06.054819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4692347-2cb1-4da9-b99b-e57b746b5f71 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:06.194037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4692347-2cb1-4da9-b99b-e57b746b5f71 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:06.233928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca788de-9f6f-4d12-9413-fa9089df6364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:06.234742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca788de-9f6f-4d12-9413-fa9089df6364 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:06.235034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca788de-9f6f-4d12-9413-fa9089df6364 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:06.235353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca788de-9f6f-4d12-9413-fa9089df6364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:06.250409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca788de-9f6f-4d12-9413-fa9089df6364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:06.251941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca788de-9f6f-4d12-9413-fa9089df6364 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-cca788de-9f6f-4d12-9413-fa9089df6364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:06.269206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:06.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:06.271405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:06.271405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:06.271405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:06.275676 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 605/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:06.277765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989abf5a-1b39-4d50-a82b-552e651de059 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:06.413322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989abf5a-1b39-4d50-a82b-552e651de059 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:06.448327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:06.455716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25983eae-1b3e-4d50-9630-82746a20997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:06.456471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25983eae-1b3e-4d50-9630-82746a20997e None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:06.456861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25983eae-1b3e-4d50-9630-82746a20997e None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:06.457463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25983eae-1b3e-4d50-9630-82746a20997e None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:06.458334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25983eae-1b3e-4d50-9630-82746a20997e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:06.458839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25983eae-1b3e-4d50-9630-82746a20997e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:06.460651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25983eae-1b3e-4d50-9630-82746a20997e None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:06.478878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 599/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:06.485019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca788de-9f6f-4d12-9413-fa9089df6364 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:06.559173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:06.612618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca788de-9f6f-4d12-9413-fa9089df6364 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:06.656637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18846b5b-d4f9-4c00-8ad1-15e69508a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:06.657324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18846b5b-d4f9-4c00-8ad1-15e69508a2ef None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:06.657686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18846b5b-d4f9-4c00-8ad1-15e69508a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:06.657992 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18846b5b-d4f9-4c00-8ad1-15e69508a2ef None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:06.658680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18846b5b-d4f9-4c00-8ad1-15e69508a2ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:06.660223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18846b5b-d4f9-4c00-8ad1-15e69508a2ef None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:06.696918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 606/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 20:12:06 2022] POST /identity/v3/users => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:12:09.237025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad0c54-9034-4f1b-80bf-74b2654db1c5 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:09.237025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad0c54-9034-4f1b-80bf-74b2654db1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:09.237025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad0c54-9034-4f1b-80bf-74b2654db1c5 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:09.237742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 600/1206] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:09.243769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9c66e-b071-4f4d-93f3-c6236c63ef58 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:09.244356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9c66e-b071-4f4d-93f3-c6236c63ef58 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:09.244356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9c66e-b071-4f4d-93f3-c6236c63ef58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:09.267057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-21a9c66e-b071-4f4d-93f3-c6236c63ef58 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:09.268271 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-21a9c66e-b071-4f4d-93f3-c6236c63ef58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:09.351134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 607/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:12:09.654836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad0c54-9034-4f1b-80bf-74b2654db1c5 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:09.774315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad0c54-9034-4f1b-80bf-74b2654db1c5 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:09.814248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a632c-e647-403f-a406-3b37a4ba7663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:09.815251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a632c-e647-403f-a406-3b37a4ba7663 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:09.815528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a632c-e647-403f-a406-3b37a4ba7663 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:09.815794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a632c-e647-403f-a406-3b37a4ba7663 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:09.828685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a632c-e647-403f-a406-3b37a4ba7663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-9f8a632c-e647-403f-a406-3b37a4ba7663 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:12:09.834471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:09.835448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:09.836277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 601/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:12:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:09.842716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9c66e-b071-4f4d-93f3-c6236c63ef58 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:09.891196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:09.963988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9c66e-b071-4f4d-93f3-c6236c63ef58 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:10.004335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c343d0-fcd1-4188-ae14-36137551fda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:10.005222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c343d0-fcd1-4188-ae14-36137551fda4 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:10.005761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c343d0-fcd1-4188-ae14-36137551fda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:10.006037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c343d0-fcd1-4188-ae14-36137551fda4 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:10.006656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c343d0-fcd1-4188-ae14-36137551fda4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:10.006963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c343d0-fcd1-4188-ae14-36137551fda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:10.008335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c343d0-fcd1-4188-ae14-36137551fda4 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:10.018285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 608/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:12:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:10.025849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a632c-e647-403f-a406-3b37a4ba7663 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:10.088414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:10.147821 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a632c-e647-403f-a406-3b37a4ba7663 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:10.189502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d548326-63ce-4397-b4ee-431464fe4d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:10.190516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d548326-63ce-4397-b4ee-431464fe4d78 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:10.190792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d548326-63ce-4397-b4ee-431464fe4d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:10.191059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d548326-63ce-4397-b4ee-431464fe4d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:10.204534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d548326-63ce-4397-b4ee-431464fe4d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:10.206393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d548326-63ce-4397-b4ee-431464fe4d78 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-2d548326-63ce-4397-b4ee-431464fe4d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:10.223019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:10.224038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:10.224929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:10.224929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:10.224929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:10.224929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 602/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:10.230572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c343d0-fcd1-4188-ae14-36137551fda4 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:10.353773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:10.357019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c343d0-fcd1-4188-ae14-36137551fda4 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:10.400731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6f422-7c06-46f4-b0c4-f6bfa97f7f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:10.402203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6f422-7c06-46f4-b0c4-f6bfa97f7f01 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:10.402507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6f422-7c06-46f4-b0c4-f6bfa97f7f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:10.402804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6f422-7c06-46f4-b0c4-f6bfa97f7f01 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:10.403512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6f422-7c06-46f4-b0c4-f6bfa97f7f01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:10.403817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6f422-7c06-46f4-b0c4-f6bfa97f7f01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:10.405234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6f422-7c06-46f4-b0c4-f6bfa97f7f01 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:10.418960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 609/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:10.425222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d548326-63ce-4397-b4ee-431464fe4d78 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:10.539575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d548326-63ce-4397-b4ee-431464fe4d78 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:10.541074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:10.585007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eef6f0-828e-41f3-813a-61f29c3c153a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:10.585887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eef6f0-828e-41f3-813a-61f29c3c153a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:10.586381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eef6f0-828e-41f3-813a-61f29c3c153a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:10.586812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eef6f0-828e-41f3-813a-61f29c3c153a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:10.596909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eef6f0-828e-41f3-813a-61f29c3c153a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:10.598759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eef6f0-828e-41f3-813a-61f29c3c153a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-63eef6f0-828e-41f3-813a-61f29c3c153a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:12:10.611791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:10.617629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:10.618836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 603/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:10.620766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6f422-7c06-46f4-b0c4-f6bfa97f7f01 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:10.728703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6f422-7c06-46f4-b0c4-f6bfa97f7f01 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:10.772969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78525783-1b64-4bb3-99c8-5f2ba4825e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:10.773710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78525783-1b64-4bb3-99c8-5f2ba4825e0b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:10.773998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78525783-1b64-4bb3-99c8-5f2ba4825e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:10.774412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78525783-1b64-4bb3-99c8-5f2ba4825e0b None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:10.775038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78525783-1b64-4bb3-99c8-5f2ba4825e0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:10.775421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78525783-1b64-4bb3-99c8-5f2ba4825e0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:10.777387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78525783-1b64-4bb3-99c8-5f2ba4825e0b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:10.797629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 610/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 20:12:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:10.800889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eef6f0-828e-41f3-813a-61f29c3c153a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:10.919854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eef6f0-828e-41f3-813a-61f29c3c153a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:10.955315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a088700-bcb5-49a4-86f6-83d44e549911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:10.955920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a088700-bcb5-49a4-86f6-83d44e549911 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:10.956193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a088700-bcb5-49a4-86f6-83d44e549911 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:10.956462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a088700-bcb5-49a4-86f6-83d44e549911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:10.988343 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:11.035994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a088700-bcb5-49a4-86f6-83d44e549911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:11.037820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a088700-bcb5-49a4-86f6-83d44e549911 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:11.153155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 604/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:12:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 05 20:12:11.159175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78525783-1b64-4bb3-99c8-5f2ba4825e0b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:11.179947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:11.272750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78525783-1b64-4bb3-99c8-5f2ba4825e0b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:11.310190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489dcbf-96b9-4d7a-9779-91c885835b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:11.310895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489dcbf-96b9-4d7a-9779-91c885835b90 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:11.311173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489dcbf-96b9-4d7a-9779-91c885835b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:11.311440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489dcbf-96b9-4d7a-9779-91c885835b90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:11.326698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489dcbf-96b9-4d7a-9779-91c885835b90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:11.328229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489dcbf-96b9-4d7a-9779-91c885835b90 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-9489dcbf-96b9-4d7a-9779-91c885835b90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:11.351354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:11.352524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:11.353575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:11.353575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:11.353575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:11.353575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 611/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:11.358380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a088700-bcb5-49a4-86f6-83d44e549911 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:11.424785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:11.470905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a088700-bcb5-49a4-86f6-83d44e549911 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:11.512534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb618fb-42f6-4e40-96c9-da9cf3c8ec33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:11.513164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb618fb-42f6-4e40-96c9-da9cf3c8ec33 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:11.513474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb618fb-42f6-4e40-96c9-da9cf3c8ec33 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:11.513758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb618fb-42f6-4e40-96c9-da9cf3c8ec33 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:11.514404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb618fb-42f6-4e40-96c9-da9cf3c8ec33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:11.514704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb618fb-42f6-4e40-96c9-da9cf3c8ec33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:11.516123 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb618fb-42f6-4e40-96c9-da9cf3c8ec33 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:11.531910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 605/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:11.537983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489dcbf-96b9-4d7a-9779-91c885835b90 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:11.629941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:11.669076 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489dcbf-96b9-4d7a-9779-91c885835b90 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:11.708055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e515226-5106-446c-a22a-60d225c6531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:11.708773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e515226-5106-446c-a22a-60d225c6531c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:11.709150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e515226-5106-446c-a22a-60d225c6531c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:11.709538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e515226-5106-446c-a22a-60d225c6531c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:11.717323 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e515226-5106-446c-a22a-60d225c6531c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:11.719257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e515226-5106-446c-a22a-60d225c6531c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-0e515226-5106-446c-a22a-60d225c6531c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:12:11.733100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:11.734373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:11.735425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:12:11.735425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:12:11.735425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:12:11.735425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:12:11.735425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:12:11.735425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:12:11.735425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:11.735425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:11.735425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 612/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:12:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:11.742223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb618fb-42f6-4e40-96c9-da9cf3c8ec33 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:11.858007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb618fb-42f6-4e40-96c9-da9cf3c8ec33 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:11.885028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:11.901536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821805ba-95b8-4cae-82e4-63f9ca879ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:11.902154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821805ba-95b8-4cae-82e4-63f9ca879ebf None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:11.902471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821805ba-95b8-4cae-82e4-63f9ca879ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:11.902825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821805ba-95b8-4cae-82e4-63f9ca879ebf None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:11.903895 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821805ba-95b8-4cae-82e4-63f9ca879ebf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:11.904231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821805ba-95b8-4cae-82e4-63f9ca879ebf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:11.906123 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821805ba-95b8-4cae-82e4-63f9ca879ebf None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:11.919796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 606/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:12:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:11.926584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e515226-5106-446c-a22a-60d225c6531c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:12.038488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e515226-5106-446c-a22a-60d225c6531c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:12.068491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:12.074979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b0501-441f-477d-b801-2b27dc13e3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:12.075574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b0501-441f-477d-b801-2b27dc13e3cd None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:12.075865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b0501-441f-477d-b801-2b27dc13e3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:12.076120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b0501-441f-477d-b801-2b27dc13e3cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:12.076946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b0501-441f-477d-b801-2b27dc13e3cd None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=5181c4b31ac240469aff7a0aee35eb7e) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:12.077195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b0501-441f-477d-b801-2b27dc13e3cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:12.078752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b0501-441f-477d-b801-2b27dc13e3cd None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:12.084556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 613/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:12:11 2022] GET /identity/v3/role_assignments?user.id=5181c4b31ac240469aff7a0aee35eb7e&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:14.324699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f53e15-06c3-4ec3-8246-e152d6961ac7 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:14.325238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f53e15-06c3-4ec3-8246-e152d6961ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:14.325238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f53e15-06c3-4ec3-8246-e152d6961ac7 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:14.329549 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 607/1220] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:14.331940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed753750-f89a-46b8-a5c8-aa5dc8ddc6db None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:14.332382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed753750-f89a-46b8-a5c8-aa5dc8ddc6db None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:14.332471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed753750-f89a-46b8-a5c8-aa5dc8ddc6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:14.351036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ed753750-f89a-46b8-a5c8-aa5dc8ddc6db None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:14.352235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ed753750-f89a-46b8-a5c8-aa5dc8ddc6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:14.420827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 614/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:12:14.719754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f53e15-06c3-4ec3-8246-e152d6961ac7 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:14.824978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f53e15-06c3-4ec3-8246-e152d6961ac7 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:14.861338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9b7fe-c742-4329-9126-ca43c00507d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:14.861990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9b7fe-c742-4329-9126-ca43c00507d8 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:14.862271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9b7fe-c742-4329-9126-ca43c00507d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:14.862538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9b7fe-c742-4329-9126-ca43c00507d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:14.874215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9b7fe-c742-4329-9126-ca43c00507d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-4dd9b7fe-c742-4329-9126-ca43c00507d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:12:14.879409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:14.880842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:14.881918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 608/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:12:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:14.888055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed753750-f89a-46b8-a5c8-aa5dc8ddc6db None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:15.005173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed753750-f89a-46b8-a5c8-aa5dc8ddc6db None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:15.033821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:15.046591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffb871-bc77-4441-8677-993551994d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:15.047195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffb871-bc77-4441-8677-993551994d27 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:15.047467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffb871-bc77-4441-8677-993551994d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:15.047759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffb871-bc77-4441-8677-993551994d27 None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:15.048409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffb871-bc77-4441-8677-993551994d27 None admin] RBAC: Adding query filter params (name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:15.048703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffb871-bc77-4441-8677-993551994d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:15.050099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffb871-bc77-4441-8677-993551994d27 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:15.059088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 615/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:12:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:15.065416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9b7fe-c742-4329-9126-ca43c00507d8 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:15.113135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:15.172005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9b7fe-c742-4329-9126-ca43c00507d8 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:15.209177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f77be-3577-49fc-8aec-f6f6e602963b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:15.209846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f77be-3577-49fc-8aec-f6f6e602963b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:15.210122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f77be-3577-49fc-8aec-f6f6e602963b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:15.210404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f77be-3577-49fc-8aec-f6f6e602963b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:15.220251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f77be-3577-49fc-8aec-f6f6e602963b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:15.221684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f77be-3577-49fc-8aec-f6f6e602963b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-ca4f77be-3577-49fc-8aec-f6f6e602963b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:15.238776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:15.239965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:15.241012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:15.241012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:15.241012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:15.241012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 609/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:15.248282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffb871-bc77-4441-8677-993551994d27 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:15.360559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffb871-bc77-4441-8677-993551994d27 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:15.405253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2b022-cebe-4d0b-8c4f-4d53d48e9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:15.406124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2b022-cebe-4d0b-8c4f-4d53d48e9a6d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:15.406417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2b022-cebe-4d0b-8c4f-4d53d48e9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:15.406897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2b022-cebe-4d0b-8c4f-4d53d48e9a6d None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:15.407668 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2b022-cebe-4d0b-8c4f-4d53d48e9a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:15.408016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2b022-cebe-4d0b-8c4f-4d53d48e9a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:15.409471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2b022-cebe-4d0b-8c4f-4d53d48e9a6d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:15.422204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 616/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:12:15.428447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f77be-3577-49fc-8aec-f6f6e602963b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:15.542862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:15.552309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f77be-3577-49fc-8aec-f6f6e602963b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:15.602055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc1767-5321-4b42-889d-85ba45281d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:15.603030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc1767-5321-4b42-889d-85ba45281d11 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:15.603574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc1767-5321-4b42-889d-85ba45281d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:15.603976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc1767-5321-4b42-889d-85ba45281d11 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:15.613047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc1767-5321-4b42-889d-85ba45281d11 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:15.615024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc1767-5321-4b42-889d-85ba45281d11 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-fbbc1767-5321-4b42-889d-85ba45281d11 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:12:15.628590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:15.629766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:15.630713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 610/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:15.636728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2b022-cebe-4d0b-8c4f-4d53d48e9a6d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:15.693361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:15.756576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2b022-cebe-4d0b-8c4f-4d53d48e9a6d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:15.796233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74ebbd-4bd3-4fae-b3d5-9bc9225fb02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:15.796869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74ebbd-4bd3-4fae-b3d5-9bc9225fb02a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:15.797148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74ebbd-4bd3-4fae-b3d5-9bc9225fb02a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:15.797421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74ebbd-4bd3-4fae-b3d5-9bc9225fb02a None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:15.798183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74ebbd-4bd3-4fae-b3d5-9bc9225fb02a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:15.798514 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74ebbd-4bd3-4fae-b3d5-9bc9225fb02a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:15.799935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74ebbd-4bd3-4fae-b3d5-9bc9225fb02a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:15.818618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 617/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 20:12:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:12:15.824566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc1767-5321-4b42-889d-85ba45281d11 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:15.933854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc1767-5321-4b42-889d-85ba45281d11 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:15.972709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835b037-4320-4d80-85ef-3700d2a605e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:15.973496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835b037-4320-4d80-85ef-3700d2a605e4 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:15.973935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835b037-4320-4d80-85ef-3700d2a605e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:15.974206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835b037-4320-4d80-85ef-3700d2a605e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:15.987118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835b037-4320-4d80-85ef-3700d2a605e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:15.988657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d835b037-4320-4d80-85ef-3700d2a605e4 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-d835b037-4320-4d80-85ef-3700d2a605e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:16.003744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:16.004989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:16.006213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:16.006213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:16.006213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:16.006213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 611/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:16.010824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74ebbd-4bd3-4fae-b3d5-9bc9225fb02a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:16.125392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74ebbd-4bd3-4fae-b3d5-9bc9225fb02a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:16.127855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:16.160873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2c9c3-a0f5-4952-b841-aae84cfaa51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:16.161504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2c9c3-a0f5-4952-b841-aae84cfaa51b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:16.161847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2c9c3-a0f5-4952-b841-aae84cfaa51b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:16.162120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2c9c3-a0f5-4952-b841-aae84cfaa51b None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:16.162763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2c9c3-a0f5-4952-b841-aae84cfaa51b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:16.163085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2c9c3-a0f5-4952-b841-aae84cfaa51b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:16.164417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2c9c3-a0f5-4952-b841-aae84cfaa51b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:16.178619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 618/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:16.184816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835b037-4320-4d80-85ef-3700d2a605e4 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:16.194798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:16.289931 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835b037-4320-4d80-85ef-3700d2a605e4 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:16.325871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532553-c454-4156-9929-bd8eba0daf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:16.326683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532553-c454-4156-9929-bd8eba0daf03 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:16.326998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532553-c454-4156-9929-bd8eba0daf03 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:16.327384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532553-c454-4156-9929-bd8eba0daf03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:16.335612 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a532553-c454-4156-9929-bd8eba0daf03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:16.337473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a532553-c454-4156-9929-bd8eba0daf03 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-8a532553-c454-4156-9929-bd8eba0daf03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:12:16.351712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:16.352847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:16.354017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:12:16.354017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:12:16.354017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:12:16.354017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:12:16.354017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:12:16.354017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:12:16.354017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:16.354017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:16.354017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 612/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:12:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:12:16.360078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2c9c3-a0f5-4952-b841-aae84cfaa51b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:16.466343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2c9c3-a0f5-4952-b841-aae84cfaa51b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:16.501351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557ae3b-21d5-43ec-9379-5869f58570f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:16.502207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557ae3b-21d5-43ec-9379-5869f58570f9 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:16.502574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557ae3b-21d5-43ec-9379-5869f58570f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:16.502886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557ae3b-21d5-43ec-9379-5869f58570f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:16.503738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557ae3b-21d5-43ec-9379-5869f58570f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:16.504090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557ae3b-21d5-43ec-9379-5869f58570f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:16.505984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557ae3b-21d5-43ec-9379-5869f58570f9 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:16.519496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 619/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:12:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:16.526007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532553-c454-4156-9929-bd8eba0daf03 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:16.602532 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:16.642845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532553-c454-4156-9929-bd8eba0daf03 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:16.680698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909366ab-dd64-4efe-aa67-f9e5438d8f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:16.681390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909366ab-dd64-4efe-aa67-f9e5438d8f92 None admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:16.681709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909366ab-dd64-4efe-aa67-f9e5438d8f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:16.681984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909366ab-dd64-4efe-aa67-f9e5438d8f92 None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/5181c4b31ac240469aff7a0aee35eb7e/roles/89ae1151daf54ba9b292e812aac0ecc6` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:16.697689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909366ab-dd64-4efe-aa67-f9e5438d8f92 None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=5181c4b31ac240469aff7a0aee35eb7e, role_id=89ae1151daf54ba9b292e812aac0ecc6)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:16.699727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909366ab-dd64-4efe-aa67-f9e5438d8f92 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:16.712498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 613/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:12:16 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/5181c4b31ac240469aff7a0aee35eb7e/roles/89ae1151daf54ba9b292e812aac0ecc6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:12:19.067710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d21a00-8e17-47d2-9a5e-4b6ae5a16e8c None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:19.068287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d21a00-8e17-47d2-9a5e-4b6ae5a16e8c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:19.068470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d21a00-8e17-47d2-9a5e-4b6ae5a16e8c None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:19.070165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 620/1233] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:19.076154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518671b3-e9dc-4273-82c7-569515d7f574 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:19.076682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518671b3-e9dc-4273-82c7-569515d7f574 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:19.076797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518671b3-e9dc-4273-82c7-569515d7f574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:19.095140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-518671b3-e9dc-4273-82c7-569515d7f574 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:19.096335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-518671b3-e9dc-4273-82c7-569515d7f574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:19.177630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 614/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:12:19.440529 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d21a00-8e17-47d2-9a5e-4b6ae5a16e8c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:19.471478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:19.565345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d21a00-8e17-47d2-9a5e-4b6ae5a16e8c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:19.606771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f698f-1b68-42f4-9d2d-628677dd652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:19.607444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f698f-1b68-42f4-9d2d-628677dd652b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:19.607788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f698f-1b68-42f4-9d2d-628677dd652b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:19.608100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f698f-1b68-42f4-9d2d-628677dd652b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:19.620430 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f698f-1b68-42f4-9d2d-628677dd652b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-973f698f-1b68-42f4-9d2d-628677dd652b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:12:19.625720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:19.627156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:19.628286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 621/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 20:12:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:19.633641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518671b3-e9dc-4273-82c7-569515d7f574 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:19.686145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:19.745722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518671b3-e9dc-4273-82c7-569515d7f574 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:19.781845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b08593-8b20-4e4b-9bb1-36d0e14b1316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:19.782484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b08593-8b20-4e4b-9bb1-36d0e14b1316 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:19.782770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b08593-8b20-4e4b-9bb1-36d0e14b1316 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:19.783072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b08593-8b20-4e4b-9bb1-36d0e14b1316 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:19.783810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b08593-8b20-4e4b-9bb1-36d0e14b1316 None admin] RBAC: Adding query filter params (name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:19.784145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b08593-8b20-4e4b-9bb1-36d0e14b1316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:19.785684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b08593-8b20-4e4b-9bb1-36d0e14b1316 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:19.794211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 615/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 20:12:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:19.800458 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f698f-1b68-42f4-9d2d-628677dd652b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:19.890309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:19.908253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f698f-1b68-42f4-9d2d-628677dd652b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:19.945539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47122074-de6e-4f30-ab2a-ceaba51254d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:19.946436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47122074-de6e-4f30-ab2a-ceaba51254d8 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:19.946886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47122074-de6e-4f30-ab2a-ceaba51254d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:19.947320 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47122074-de6e-4f30-ab2a-ceaba51254d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:19.961698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47122074-de6e-4f30-ab2a-ceaba51254d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:19.963302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47122074-de6e-4f30-ab2a-ceaba51254d8 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-47122074-de6e-4f30-ab2a-ceaba51254d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:19.979055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:19.980888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:19.981830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:19.981830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:19.981830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:19.981830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 622/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:12:19.986012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b08593-8b20-4e4b-9bb1-36d0e14b1316 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:20.094349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b08593-8b20-4e4b-9bb1-36d0e14b1316 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:20.106943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:20.136729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e847c74-d4c3-48d7-910d-98d61d1ce979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:20.137341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e847c74-d4c3-48d7-910d-98d61d1ce979 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:20.137657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e847c74-d4c3-48d7-910d-98d61d1ce979 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:20.137923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e847c74-d4c3-48d7-910d-98d61d1ce979 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:20.138560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e847c74-d4c3-48d7-910d-98d61d1ce979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:20.138855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e847c74-d4c3-48d7-910d-98d61d1ce979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:20.140216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e847c74-d4c3-48d7-910d-98d61d1ce979 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:20.152147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 616/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:12:20.158005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47122074-de6e-4f30-ab2a-ceaba51254d8 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:20.264904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47122074-de6e-4f30-ab2a-ceaba51254d8 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:20.301345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34adca1-4777-4f6f-9670-c48571f2e215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:20.302277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34adca1-4777-4f6f-9670-c48571f2e215 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:20.302496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34adca1-4777-4f6f-9670-c48571f2e215 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:20.302813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34adca1-4777-4f6f-9670-c48571f2e215 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:20.306213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:20.312592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34adca1-4777-4f6f-9670-c48571f2e215 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:20.314309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34adca1-4777-4f6f-9670-c48571f2e215 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-d34adca1-4777-4f6f-9670-c48571f2e215 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:12:20.327567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:20.328772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:20.329958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 623/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:20.334616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e847c74-d4c3-48d7-910d-98d61d1ce979 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:20.439070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e847c74-d4c3-48d7-910d-98d61d1ce979 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:20.476230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5632ee-9ae0-4f46-ac94-377a5a0eab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:20.476915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5632ee-9ae0-4f46-ac94-377a5a0eab94 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:20.477185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5632ee-9ae0-4f46-ac94-377a5a0eab94 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:20.477493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5632ee-9ae0-4f46-ac94-377a5a0eab94 None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:20.478395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5632ee-9ae0-4f46-ac94-377a5a0eab94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:20.478725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5632ee-9ae0-4f46-ac94-377a5a0eab94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:20.480340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5632ee-9ae0-4f46-ac94-377a5a0eab94 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:20.498331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 617/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 20:12:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:20.505119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34adca1-4777-4f6f-9670-c48571f2e215 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:20.615655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34adca1-4777-4f6f-9670-c48571f2e215 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:20.650778 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bc651-52d0-40cf-a550-acab558d711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:20.651949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bc651-52d0-40cf-a550-acab558d711a None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:20.652229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bc651-52d0-40cf-a550-acab558d711a None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:20.652501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bc651-52d0-40cf-a550-acab558d711a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:20.719493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:20.726742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bc651-52d0-40cf-a550-acab558d711a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:20.728198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bc651-52d0-40cf-a550-acab558d711a None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:20.834402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 624/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:12:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:12:20.842845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5632ee-9ae0-4f46-ac94-377a5a0eab94 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:20.875063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:20.958178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5632ee-9ae0-4f46-ac94-377a5a0eab94 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:20.996044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b8342-46ae-4545-b5d9-3d8f24bd5d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:20.996581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b8342-46ae-4545-b5d9-3d8f24bd5d99 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:20.996856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b8342-46ae-4545-b5d9-3d8f24bd5d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:20.997140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b8342-46ae-4545-b5d9-3d8f24bd5d99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:21.007437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b8342-46ae-4545-b5d9-3d8f24bd5d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:21.008852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b8342-46ae-4545-b5d9-3d8f24bd5d99 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-841b8342-46ae-4545-b5d9-3d8f24bd5d99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:21.024127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:21.025191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:21.027080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:21.027080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:21.027080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:21.027080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 618/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:21.032305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bc651-52d0-40cf-a550-acab558d711a None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:21.137388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:21.143495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bc651-52d0-40cf-a550-acab558d711a None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:21.181189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775c4e0-5049-41cd-addf-e7d97ec3774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:21.181865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775c4e0-5049-41cd-addf-e7d97ec3774d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:21.182079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775c4e0-5049-41cd-addf-e7d97ec3774d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:21.183886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775c4e0-5049-41cd-addf-e7d97ec3774d None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:21.184531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775c4e0-5049-41cd-addf-e7d97ec3774d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:21.184832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775c4e0-5049-41cd-addf-e7d97ec3774d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:21.186218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775c4e0-5049-41cd-addf-e7d97ec3774d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:21.204754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 625/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:12:21.206044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b8342-46ae-4545-b5d9-3d8f24bd5d99 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:21.292222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:21.319197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b8342-46ae-4545-b5d9-3d8f24bd5d99 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:21.356082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3d603-c187-4c05-ab8e-ea35949a4545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:21.356710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3d603-c187-4c05-ab8e-ea35949a4545 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:21.357171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3d603-c187-4c05-ab8e-ea35949a4545 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:21.357435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3d603-c187-4c05-ab8e-ea35949a4545 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:21.365330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3d603-c187-4c05-ab8e-ea35949a4545 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:21.367051 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3d603-c187-4c05-ab8e-ea35949a4545 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-f4a3d603-c187-4c05-ab8e-ea35949a4545 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:12:21.382686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:21.383914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:21.384795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:12:21.384795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:12:21.384795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:12:21.384795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:12:21.384795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:12:21.384795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:12:21.384795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:21.384795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:21.384795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 619/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:12:21.390683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775c4e0-5049-41cd-addf-e7d97ec3774d None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:21.501849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775c4e0-5049-41cd-addf-e7d97ec3774d None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:21.538521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31f43b-9c10-419e-8e90-23fcd113e03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:21.539184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31f43b-9c10-419e-8e90-23fcd113e03d None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:21.539460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31f43b-9c10-419e-8e90-23fcd113e03d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:21.539727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31f43b-9c10-419e-8e90-23fcd113e03d None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:21.540411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31f43b-9c10-419e-8e90-23fcd113e03d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:21.540711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31f43b-9c10-419e-8e90-23fcd113e03d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:21.542215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31f43b-9c10-419e-8e90-23fcd113e03d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:21.555456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 626/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:12:21.562419 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3d603-c187-4c05-ab8e-ea35949a4545 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:21.668300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3d603-c187-4c05-ab8e-ea35949a4545 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:21.707967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b0009-9426-4983-8d6c-feab7b241d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:21.708603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b0009-9426-4983-8d6c-feab7b241d3d None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:21.708985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b0009-9426-4983-8d6c-feab7b241d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:21.709323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b0009-9426-4983-8d6c-feab7b241d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:21.710109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b0009-9426-4983-8d6c-feab7b241d3d None admin] RBAC: Adding query filter params (role.id=89ae1151daf54ba9b292e812aac0ecc6, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=5181c4b31ac240469aff7a0aee35eb7e) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:21.710515 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b0009-9426-4983-8d6c-feab7b241d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:21.712054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b0009-9426-4983-8d6c-feab7b241d3d None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:21.718329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 620/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:12:21 2022] GET /identity/v3/role_assignments?user.id=5181c4b31ac240469aff7a0aee35eb7e&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=89ae1151daf54ba9b292e812aac0ecc6 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:12:24.259353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bb877-6dfb-43f6-a960-ed943410ecb4 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:24.259896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bb877-6dfb-43f6-a960-ed943410ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:24.259896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bb877-6dfb-43f6-a960-ed943410ecb4 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:24.261165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 627/1247] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:24.266723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2090f-dba2-402b-8939-7135fc0fedf3 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:24.267162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2090f-dba2-402b-8939-7135fc0fedf3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:24.267270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2090f-dba2-402b-8939-7135fc0fedf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:24.274474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:24.288485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f1a2090f-dba2-402b-8939-7135fc0fedf3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:24.289894 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f1a2090f-dba2-402b-8939-7135fc0fedf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:24.360089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 621/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:12:24.629119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bb877-6dfb-43f6-a960-ed943410ecb4 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:24.646711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:24.737577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bb877-6dfb-43f6-a960-ed943410ecb4 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:24.775581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641d5ae-a8bf-4fd5-97b8-108190a3aca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:24.776839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641d5ae-a8bf-4fd5-97b8-108190a3aca5 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:24.777097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641d5ae-a8bf-4fd5-97b8-108190a3aca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:24.777365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641d5ae-a8bf-4fd5-97b8-108190a3aca5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:24.788780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641d5ae-a8bf-4fd5-97b8-108190a3aca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-9641d5ae-a8bf-4fd5-97b8-108190a3aca5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:12:24.793559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:24.794665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:24.795626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 628/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:12:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:24.800873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2090f-dba2-402b-8939-7135fc0fedf3 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:24.904022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2090f-dba2-402b-8939-7135fc0fedf3 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:24.935187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:24.940948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3b530-077e-4cc4-8aef-43bbeae1c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:24.941584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3b530-077e-4cc4-8aef-43bbeae1c744 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:24.941865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3b530-077e-4cc4-8aef-43bbeae1c744 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:24.942137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3b530-077e-4cc4-8aef-43bbeae1c744 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:24.942797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3b530-077e-4cc4-8aef-43bbeae1c744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:24.943098 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3b530-077e-4cc4-8aef-43bbeae1c744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:24.944462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3b530-077e-4cc4-8aef-43bbeae1c744 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:24.953969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 622/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:12:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:24.959638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641d5ae-a8bf-4fd5-97b8-108190a3aca5 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:25.045072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:25.070570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641d5ae-a8bf-4fd5-97b8-108190a3aca5 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:25.113114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc724a-d744-4e9b-8296-c10ebd2e8ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:25.113769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc724a-d744-4e9b-8296-c10ebd2e8ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:25.114062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc724a-d744-4e9b-8296-c10ebd2e8ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:25.114321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc724a-d744-4e9b-8296-c10ebd2e8ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:25.124315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc724a-d744-4e9b-8296-c10ebd2e8ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:25.125789 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc724a-d744-4e9b-8296-c10ebd2e8ce2 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-ecbc724a-d744-4e9b-8296-c10ebd2e8ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:25.141155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:25.142537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:25.144107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:25.144107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:25.144107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:25.144107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 629/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:25.148433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3b530-077e-4cc4-8aef-43bbeae1c744 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:25.255811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3b530-077e-4cc4-8aef-43bbeae1c744 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:25.294095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778b74c-448d-4d9e-afcb-da24b7bd22bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:25.294967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778b74c-448d-4d9e-afcb-da24b7bd22bd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:25.294967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778b74c-448d-4d9e-afcb-da24b7bd22bd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:25.295236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778b74c-448d-4d9e-afcb-da24b7bd22bd None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:25.295881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778b74c-448d-4d9e-afcb-da24b7bd22bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:25.296145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778b74c-448d-4d9e-afcb-da24b7bd22bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:25.297552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778b74c-448d-4d9e-afcb-da24b7bd22bd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:25.310178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 623/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:12:25.316207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc724a-d744-4e9b-8296-c10ebd2e8ce2 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:25.421305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc724a-d744-4e9b-8296-c10ebd2e8ce2 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:25.449977 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:25.458351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4835c-0b3e-4a4d-80df-3d1f4a2de7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:25.458992 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4835c-0b3e-4a4d-80df-3d1f4a2de7fe None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:25.459265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4835c-0b3e-4a4d-80df-3d1f4a2de7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:25.459528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4835c-0b3e-4a4d-80df-3d1f4a2de7fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:25.467976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4835c-0b3e-4a4d-80df-3d1f4a2de7fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:25.469411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4835c-0b3e-4a4d-80df-3d1f4a2de7fe None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-55e4835c-0b3e-4a4d-80df-3d1f4a2de7fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:12:25.481153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:25.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:25.483344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 630/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 20:12:25.488433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778b74c-448d-4d9e-afcb-da24b7bd22bd None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:25.548432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:25.602951 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778b74c-448d-4d9e-afcb-da24b7bd22bd None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:25.638750 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12b242-dbb1-4fa1-b8aa-5e619bef74af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:25.639367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12b242-dbb1-4fa1-b8aa-5e619bef74af None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:25.639647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12b242-dbb1-4fa1-b8aa-5e619bef74af None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:25.639918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12b242-dbb1-4fa1-b8aa-5e619bef74af None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:25.640650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12b242-dbb1-4fa1-b8aa-5e619bef74af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:25.640955 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12b242-dbb1-4fa1-b8aa-5e619bef74af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:25.642439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12b242-dbb1-4fa1-b8aa-5e619bef74af None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:25.666420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 624/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 20:12:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:25.668194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4835c-0b3e-4a4d-80df-3d1f4a2de7fe None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:25.771089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4835c-0b3e-4a4d-80df-3d1f4a2de7fe None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:25.809159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53d16f-80af-4b8a-8479-b3551e06b349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:25.809719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53d16f-80af-4b8a-8479-b3551e06b349 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:25.809991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53d16f-80af-4b8a-8479-b3551e06b349 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:25.810255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53d16f-80af-4b8a-8479-b3551e06b349 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:25.865138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:25.880944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53d16f-80af-4b8a-8479-b3551e06b349 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:25.882468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53d16f-80af-4b8a-8479-b3551e06b349 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:25.983389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 631/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:12:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:12:25.989150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12b242-dbb1-4fa1-b8aa-5e619bef74af None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:26.086170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12b242-dbb1-4fa1-b8aa-5e619bef74af None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:26.098990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:26.120104 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1930-8dc6-4d8c-9949-7c395642675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:26.120722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1930-8dc6-4d8c-9949-7c395642675a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:26.121001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1930-8dc6-4d8c-9949-7c395642675a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:26.121273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1930-8dc6-4d8c-9949-7c395642675a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:26.130850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df1930-8dc6-4d8c-9949-7c395642675a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:26.132285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df1930-8dc6-4d8c-9949-7c395642675a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-74df1930-8dc6-4d8c-9949-7c395642675a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:26.146239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:26.147301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:26.148363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:26.148363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:26.148363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:26.148363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 625/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:26.153717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53d16f-80af-4b8a-8479-b3551e06b349 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:26.237870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:26.249761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53d16f-80af-4b8a-8479-b3551e06b349 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:26.282976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408d7e4-d1e6-4d13-b729-a4f9a241947b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:26.283568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408d7e4-d1e6-4d13-b729-a4f9a241947b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:26.283845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408d7e4-d1e6-4d13-b729-a4f9a241947b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:26.284115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408d7e4-d1e6-4d13-b729-a4f9a241947b None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:26.284742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408d7e4-d1e6-4d13-b729-a4f9a241947b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:26.285040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408d7e4-d1e6-4d13-b729-a4f9a241947b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:26.286410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408d7e4-d1e6-4d13-b729-a4f9a241947b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:26.297197 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 632/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:26.302812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1930-8dc6-4d8c-9949-7c395642675a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:26.398286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1930-8dc6-4d8c-9949-7c395642675a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:26.430364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44194868-ec04-4f46-bdbe-1c43378658ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:26.430961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44194868-ec04-4f46-bdbe-1c43378658ec None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:26.431235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44194868-ec04-4f46-bdbe-1c43378658ec None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:26.431501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44194868-ec04-4f46-bdbe-1c43378658ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:26.437870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44194868-ec04-4f46-bdbe-1c43378658ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:26.439295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44194868-ec04-4f46-bdbe-1c43378658ec None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-44194868-ec04-4f46-bdbe-1c43378658ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:12:26.450832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:26.451835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:26.457314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:12:26.457314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:12:26.457314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:12:26.457314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:12:26.457314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:12:26.457314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:12:26.457314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:26.457314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:26.457314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 626/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:12:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:26.457964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408d7e4-d1e6-4d13-b729-a4f9a241947b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:26.554733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408d7e4-d1e6-4d13-b729-a4f9a241947b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:26.592394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740835d0-344e-41ad-90f6-1772cbf22bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:26.593078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740835d0-344e-41ad-90f6-1772cbf22bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:26.593372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740835d0-344e-41ad-90f6-1772cbf22bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:26.593659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740835d0-344e-41ad-90f6-1772cbf22bd3 None admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:26.594339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740835d0-344e-41ad-90f6-1772cbf22bd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:26.594636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740835d0-344e-41ad-90f6-1772cbf22bd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:26.596051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740835d0-344e-41ad-90f6-1772cbf22bd3 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:26.608515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 633/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:12:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:26.614565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44194868-ec04-4f46-bdbe-1c43378658ec None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:26.639757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:26.712592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44194868-ec04-4f46-bdbe-1c43378658ec None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:26.743827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94906004-9b4f-4353-be99-ea652d94cecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:26.744414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94906004-9b4f-4353-be99-ea652d94cecd None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:26.744694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94906004-9b4f-4353-be99-ea652d94cecd None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:26.744966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94906004-9b4f-4353-be99-ea652d94cecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:26.745707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94906004-9b4f-4353-be99-ea652d94cecd None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=5181c4b31ac240469aff7a0aee35eb7e) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:26.746524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94906004-9b4f-4353-be99-ea652d94cecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:26.747941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94906004-9b4f-4353-be99-ea652d94cecd None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:26.753054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 627/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:12:26 2022] GET /identity/v3/role_assignments?user.id=5181c4b31ac240469aff7a0aee35eb7e&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=36331ec562ff42f883fc883363bc4c8d => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:28.853903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd1af5-8528-40de-898a-8db84501cfdc None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:28.854453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd1af5-8528-40de-898a-8db84501cfdc None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:28.854453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd1af5-8528-40de-898a-8db84501cfdc None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:28.855794 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 634/1261] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:28.860792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59523b14-aaee-4a33-8c75-5db52710a70c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:28.861207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59523b14-aaee-4a33-8c75-5db52710a70c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:28.861337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59523b14-aaee-4a33-8c75-5db52710a70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:28.879739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-59523b14-aaee-4a33-8c75-5db52710a70c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:28.880928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-59523b14-aaee-4a33-8c75-5db52710a70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:28.945856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 628/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:12:29.208350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd1af5-8528-40de-898a-8db84501cfdc None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:29.219597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:29.306681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd1af5-8528-40de-898a-8db84501cfdc None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:29.338934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa72fc-071b-4744-8f61-541181e01445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:29.339536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa72fc-071b-4744-8f61-541181e01445 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:29.340352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa72fc-071b-4744-8f61-541181e01445 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:29.340617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa72fc-071b-4744-8f61-541181e01445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:29.350359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa72fc-071b-4744-8f61-541181e01445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-cbfa72fc-071b-4744-8f61-541181e01445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:12:29.354971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:29.356051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:29.356911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 635/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:12:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:29.362049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59523b14-aaee-4a33-8c75-5db52710a70c None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:29.379960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:29.457973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59523b14-aaee-4a33-8c75-5db52710a70c None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:29.489761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764cdfd-6191-4802-9e6b-02940cb46df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:29.490295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764cdfd-6191-4802-9e6b-02940cb46df3 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:29.490567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764cdfd-6191-4802-9e6b-02940cb46df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:29.490833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764cdfd-6191-4802-9e6b-02940cb46df3 None admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:29.491479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f764cdfd-6191-4802-9e6b-02940cb46df3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:29.491774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f764cdfd-6191-4802-9e6b-02940cb46df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:29.493117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f764cdfd-6191-4802-9e6b-02940cb46df3 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:29.500662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 629/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:12:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:29.506299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa72fc-071b-4744-8f61-541181e01445 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:29.578635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:29.618194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa72fc-071b-4744-8f61-541181e01445 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:29.655113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa47ea1-c02c-474f-a4c0-9862c83fee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:29.655733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa47ea1-c02c-474f-a4c0-9862c83fee89 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:29.656007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa47ea1-c02c-474f-a4c0-9862c83fee89 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:29.656273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa47ea1-c02c-474f-a4c0-9862c83fee89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:29.665867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa47ea1-c02c-474f-a4c0-9862c83fee89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:29.667293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa47ea1-c02c-474f-a4c0-9862c83fee89 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-0aa47ea1-c02c-474f-a4c0-9862c83fee89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:29.682021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:29.683104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:29.684855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:29.684855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:29.684855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:29.684855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 636/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:29.689449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764cdfd-6191-4802-9e6b-02940cb46df3 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:29.768623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:29.787117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764cdfd-6191-4802-9e6b-02940cb46df3 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:29.819352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e87857-24dc-438b-8426-64f0734acc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:29.819940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e87857-24dc-438b-8426-64f0734acc71 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:29.820214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e87857-24dc-438b-8426-64f0734acc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:29.820484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e87857-24dc-438b-8426-64f0734acc71 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:29.821109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e87857-24dc-438b-8426-64f0734acc71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:29.821407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e87857-24dc-438b-8426-64f0734acc71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:29.822791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e87857-24dc-438b-8426-64f0734acc71 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:29.834461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 630/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:29.840014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa47ea1-c02c-474f-a4c0-9862c83fee89 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:29.937285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa47ea1-c02c-474f-a4c0-9862c83fee89 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:29.956832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:29.970991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3966-b5a1-4c9c-b3c7-16726aff3d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:29.971610 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3966-b5a1-4c9c-b3c7-16726aff3d20 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:29.971892 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3966-b5a1-4c9c-b3c7-16726aff3d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:29.972163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3966-b5a1-4c9c-b3c7-16726aff3d20 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:29.979405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd3966-b5a1-4c9c-b3c7-16726aff3d20 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:29.981352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd3966-b5a1-4c9c-b3c7-16726aff3d20 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-b2bd3966-b5a1-4c9c-b3c7-16726aff3d20 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:12:29.992153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:29.993236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:29.994394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 637/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:29.999560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e87857-24dc-438b-8426-64f0734acc71 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:30.093714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e87857-24dc-438b-8426-64f0734acc71 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:30.126413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4d8b8-5df3-40d4-b5bd-a44a2e7c815a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:30.126998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4d8b8-5df3-40d4-b5bd-a44a2e7c815a None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:30.127273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4d8b8-5df3-40d4-b5bd-a44a2e7c815a None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:30.127541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4d8b8-5df3-40d4-b5bd-a44a2e7c815a None admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:30.128249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4d8b8-5df3-40d4-b5bd-a44a2e7c815a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:30.128548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4d8b8-5df3-40d4-b5bd-a44a2e7c815a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:30.129992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a4d8b8-5df3-40d4-b5bd-a44a2e7c815a None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:30.138236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:30.146741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 631/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 20:12:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:30.152374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3966-b5a1-4c9c-b3c7-16726aff3d20 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:30.250179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3966-b5a1-4c9c-b3c7-16726aff3d20 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:30.281473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0882c9-58ee-4ec3-884d-a5151d6fc753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:30.282113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0882c9-58ee-4ec3-884d-a5151d6fc753 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:30.282389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0882c9-58ee-4ec3-884d-a5151d6fc753 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:30.282659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0882c9-58ee-4ec3-884d-a5151d6fc753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:30.292683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0882c9-58ee-4ec3-884d-a5151d6fc753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:30.294662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0882c9-58ee-4ec3-884d-a5151d6fc753 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-5b0882c9-58ee-4ec3-884d-a5151d6fc753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:30.308144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:30.309839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:30.310746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:30.310746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:30.310746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:30.310746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 638/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:30.315655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4d8b8-5df3-40d4-b5bd-a44a2e7c815a None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:30.410118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4d8b8-5df3-40d4-b5bd-a44a2e7c815a None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:30.443333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498d50d-e924-439b-aaa5-fea6fedcced2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:30.443920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498d50d-e924-439b-aaa5-fea6fedcced2 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:30.444192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498d50d-e924-439b-aaa5-fea6fedcced2 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:30.444460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498d50d-e924-439b-aaa5-fea6fedcced2 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:30.445087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498d50d-e924-439b-aaa5-fea6fedcced2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:30.445383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498d50d-e924-439b-aaa5-fea6fedcced2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:30.446755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498d50d-e924-439b-aaa5-fea6fedcced2 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:30.457337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 632/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:30.462816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0882c9-58ee-4ec3-884d-a5151d6fc753 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:30.494007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:30.563323 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0882c9-58ee-4ec3-884d-a5151d6fc753 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:30.598451 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166e86-69d7-4139-83b8-5ac03bdc0db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:30.599064 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166e86-69d7-4139-83b8-5ac03bdc0db0 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:30.599335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166e86-69d7-4139-83b8-5ac03bdc0db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:30.599598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166e86-69d7-4139-83b8-5ac03bdc0db0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:30.606521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3166e86-69d7-4139-83b8-5ac03bdc0db0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:30.607934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3166e86-69d7-4139-83b8-5ac03bdc0db0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-d3166e86-69d7-4139-83b8-5ac03bdc0db0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:12:30.618649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:30.619680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:30.620764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:12:30.620764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:12:30.620764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:12:30.620764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:12:30.620764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:12:30.620764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:12:30.620764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:30.620764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:30.620764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 639/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:12:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:30.625504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498d50d-e924-439b-aaa5-fea6fedcced2 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:30.681455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:30.721650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498d50d-e924-439b-aaa5-fea6fedcced2 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:30.753407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd454f82-3c5d-43ea-8221-efcd524d0d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:30.754020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd454f82-3c5d-43ea-8221-efcd524d0d30 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:30.754295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd454f82-3c5d-43ea-8221-efcd524d0d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:30.754561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd454f82-3c5d-43ea-8221-efcd524d0d30 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:30.755233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd454f82-3c5d-43ea-8221-efcd524d0d30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:30.755531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd454f82-3c5d-43ea-8221-efcd524d0d30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:30.756942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd454f82-3c5d-43ea-8221-efcd524d0d30 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:30.768891 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 633/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:30.774666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166e86-69d7-4139-83b8-5ac03bdc0db0 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:30.863177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:30.872401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166e86-69d7-4139-83b8-5ac03bdc0db0 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:30.905935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0d22d-d1a9-47ec-b2ef-ba414d8882a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:30.906578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0d22d-d1a9-47ec-b2ef-ba414d8882a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:30.906854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0d22d-d1a9-47ec-b2ef-ba414d8882a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:30.907125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0d22d-d1a9-47ec-b2ef-ba414d8882a0 None admin] PATH_INFO: `/v3/projects/3130efb60669479aae97cf2ec518e76d/users/5181c4b31ac240469aff7a0aee35eb7e/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:30.909006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0d22d-d1a9-47ec-b2ef-ba414d8882a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3130efb60669479aae97cf2ec518e76d, user_id=5181c4b31ac240469aff7a0aee35eb7e, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:30.910445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0d22d-d1a9-47ec-b2ef-ba414d8882a0 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:30.922812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 640/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 20:12:30 2022] PUT /identity/v3/projects/3130efb60669479aae97cf2ec518e76d/users/5181c4b31ac240469aff7a0aee35eb7e/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:12:33.026289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d966f9-2abd-4d99-a0c5-f13ed468fea8 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:33.026831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d966f9-2abd-4d99-a0c5-f13ed468fea8 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:33.026831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d966f9-2abd-4d99-a0c5-f13ed468fea8 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:33.028143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 634/1274] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:33.033359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9d49-cae9-433e-b933-ad402c108753 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:33.033830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9d49-cae9-433e-b933-ad402c108753 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:33.033939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9d49-cae9-433e-b933-ad402c108753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:33.052773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-46bd9d49-cae9-433e-b933-ad402c108753 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:33.053987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-46bd9d49-cae9-433e-b933-ad402c108753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:33.119550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 641/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:12:33.388327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d966f9-2abd-4d99-a0c5-f13ed468fea8 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:33.485214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d966f9-2abd-4d99-a0c5-f13ed468fea8 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:33.501455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:33.518055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc971a8b-acbd-45fb-86dc-2f5d7a701ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:33.518671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc971a8b-acbd-45fb-86dc-2f5d7a701ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:33.518942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc971a8b-acbd-45fb-86dc-2f5d7a701ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:33.519208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc971a8b-acbd-45fb-86dc-2f5d7a701ff6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:33.529534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc971a8b-acbd-45fb-86dc-2f5d7a701ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-bc971a8b-acbd-45fb-86dc-2f5d7a701ff6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 20:12:33.534147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:33.535226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:33.536377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 635/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 20:12:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:33.541130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9d49-cae9-433e-b933-ad402c108753 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:33.633244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:33.651231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9d49-cae9-433e-b933-ad402c108753 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:33.684133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d1d0-c75d-47b9-8833-11307ab7986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:33.684731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d1d0-c75d-47b9-8833-11307ab7986c None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:33.685006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d1d0-c75d-47b9-8833-11307ab7986c None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:33.685274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d1d0-c75d-47b9-8833-11307ab7986c None admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:33.685969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8d1d0-c75d-47b9-8833-11307ab7986c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:33.686253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8d1d0-c75d-47b9-8833-11307ab7986c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:33.687597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8d1d0-c75d-47b9-8833-11307ab7986c None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:33.695553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 642/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 20:12:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:33.701391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc971a8b-acbd-45fb-86dc-2f5d7a701ff6 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:33.795869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc971a8b-acbd-45fb-86dc-2f5d7a701ff6 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:33.828148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f9e9c-119e-4293-9a5b-b428e3be2a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:33.828751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f9e9c-119e-4293-9a5b-b428e3be2a8f None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:33.829025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f9e9c-119e-4293-9a5b-b428e3be2a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:33.829292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f9e9c-119e-4293-9a5b-b428e3be2a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:33.838787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f9e9c-119e-4293-9a5b-b428e3be2a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:33.840827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f9e9c-119e-4293-9a5b-b428e3be2a8f None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-743f9e9c-119e-4293-9a5b-b428e3be2a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:33.854555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:33.855713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:33.856790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:33.856790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:33.856790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:33.856790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 636/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:33.861176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d1d0-c75d-47b9-8833-11307ab7986c None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:33.959314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d1d0-c75d-47b9-8833-11307ab7986c None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:33.990936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae052e-52a9-45dc-85d5-0b2dcb171be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:33.991523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae052e-52a9-45dc-85d5-0b2dcb171be3 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:33.991795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae052e-52a9-45dc-85d5-0b2dcb171be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:33.992061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae052e-52a9-45dc-85d5-0b2dcb171be3 None admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:33.992678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae052e-52a9-45dc-85d5-0b2dcb171be3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:33.992972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae052e-52a9-45dc-85d5-0b2dcb171be3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:33.994335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae052e-52a9-45dc-85d5-0b2dcb171be3 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:34.005416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 643/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:34.011256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f9e9c-119e-4293-9a5b-b428e3be2a8f None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:34.041374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:34.107604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f9e9c-119e-4293-9a5b-b428e3be2a8f None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:34.139777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a369e98-ea28-4acb-9c97-99cddd7ce31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:34.140378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a369e98-ea28-4acb-9c97-99cddd7ce31c None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:34.140650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a369e98-ea28-4acb-9c97-99cddd7ce31c None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:34.140917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a369e98-ea28-4acb-9c97-99cddd7ce31c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:34.148587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a369e98-ea28-4acb-9c97-99cddd7ce31c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:34.150031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a369e98-ea28-4acb-9c97-99cddd7ce31c None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-8a369e98-ea28-4acb-9c97-99cddd7ce31c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 20:12:34.160565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:34.161664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:34.162782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 637/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:34.167367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae052e-52a9-45dc-85d5-0b2dcb171be3 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:34.171057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:34.265025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae052e-52a9-45dc-85d5-0b2dcb171be3 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:34.296781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56339ba0-cb70-4649-8176-a70f8edc926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:34.297360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56339ba0-cb70-4649-8176-a70f8edc926b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:34.297665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56339ba0-cb70-4649-8176-a70f8edc926b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:34.297938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56339ba0-cb70-4649-8176-a70f8edc926b None admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:34.298651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56339ba0-cb70-4649-8176-a70f8edc926b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:34.298951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56339ba0-cb70-4649-8176-a70f8edc926b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:34.300352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56339ba0-cb70-4649-8176-a70f8edc926b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:34.317474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 644/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 20:12:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:34.323137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a369e98-ea28-4acb-9c97-99cddd7ce31c None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:34.409131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:34.418908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a369e98-ea28-4acb-9c97-99cddd7ce31c None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:34.451845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e61b2-4887-4957-9b10-981851df354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:34.452430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e61b2-4887-4957-9b10-981851df354b None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:34.452708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e61b2-4887-4957-9b10-981851df354b None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:34.452974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e61b2-4887-4957-9b10-981851df354b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:34.516517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e61b2-4887-4957-9b10-981851df354b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:34.517915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e61b2-4887-4957-9b10-981851df354b None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:34.602668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:34.612923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 638/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 20:12:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 20:12:34.618719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56339ba0-cb70-4649-8176-a70f8edc926b None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:34.677524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:34.715007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56339ba0-cb70-4649-8176-a70f8edc926b None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:34.747931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b945043-896d-42ef-9b41-68a8af298f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:34.748539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b945043-896d-42ef-9b41-68a8af298f89 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:34.748815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b945043-896d-42ef-9b41-68a8af298f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:34.749081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b945043-896d-42ef-9b41-68a8af298f89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:34.758651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b945043-896d-42ef-9b41-68a8af298f89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:34.760065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b945043-896d-42ef-9b41-68a8af298f89 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-6b945043-896d-42ef-9b41-68a8af298f89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 20:12:34.774907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:34.775969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 20:12:34.777061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 20:12:34.777061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 20:12:34.777061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:34.777061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 645/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:34.781657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e61b2-4887-4957-9b10-981851df354b None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:34.879006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e61b2-4887-4957-9b10-981851df354b None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:34.911895 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e84ae-3c66-4072-9663-93d84f1ea1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:34.912486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e84ae-3c66-4072-9663-93d84f1ea1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:34.912756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e84ae-3c66-4072-9663-93d84f1ea1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:34.913023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e84ae-3c66-4072-9663-93d84f1ea1f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:34.913670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e84ae-3c66-4072-9663-93d84f1ea1f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:34.913964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e84ae-3c66-4072-9663-93d84f1ea1f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:34.915333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e84ae-3c66-4072-9663-93d84f1ea1f0 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:34.926522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 639/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 20:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:34.932111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b945043-896d-42ef-9b41-68a8af298f89 None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:35.027302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b945043-896d-42ef-9b41-68a8af298f89 None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:35.040590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:35.060784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6649-b4cc-46e4-a2b9-c6018d7e48df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:35.061395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6649-b4cc-46e4-a2b9-c6018d7e48df None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:35.061710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6649-b4cc-46e4-a2b9-c6018d7e48df None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:35.061981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf6649-b4cc-46e4-a2b9-c6018d7e48df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:35.068607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6649-b4cc-46e4-a2b9-c6018d7e48df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:35.070062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf6649-b4cc-46e4-a2b9-c6018d7e48df None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application [None req-46cf6649-b4cc-46e4-a2b9-c6018d7e48df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 20:12:35.080876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:35.081922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:35.082981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 20:12:35.082981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 20:12:35.082981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 20:12:35.082981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 20:12:35.082981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 20:12:35.082981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 20:12:35.082981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 20:12:35.082981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: ERROR keystone.server.flask.application Oct 05 20:12:35.082981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 646/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 20:12:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:35.087663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e84ae-3c66-4072-9663-93d84f1ea1f0 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:35.137011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:35.184859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e84ae-3c66-4072-9663-93d84f1ea1f0 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:35.217145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9c7a9-2ec7-4897-a3aa-8a8b4c2b5d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:35.217756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9c7a9-2ec7-4897-a3aa-8a8b4c2b5d06 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:35.218030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9c7a9-2ec7-4897-a3aa-8a8b4c2b5d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:35.218295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9c7a9-2ec7-4897-a3aa-8a8b4c2b5d06 None admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:35.218963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9c7a9-2ec7-4897-a3aa-8a8b4c2b5d06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:35.219258 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9c7a9-2ec7-4897-a3aa-8a8b4c2b5d06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:35.220649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9c7a9-2ec7-4897-a3aa-8a8b4c2b5d06 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:35.232490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 640/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 20:12:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:35.238509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6649-b4cc-46e4-a2b9-c6018d7e48df None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:35.335066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf6649-b4cc-46e4-a2b9-c6018d7e48df None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:35.366981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2455fd-f927-4391-b1b8-49997ac28e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:35.367564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2455fd-f927-4391-b1b8-49997ac28e34 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:35.367840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2455fd-f927-4391-b1b8-49997ac28e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:35.368108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2455fd-f927-4391-b1b8-49997ac28e34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:35.368817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2455fd-f927-4391-b1b8-49997ac28e34 None admin] RBAC: Adding query filter params (role.id=36331ec562ff42f883fc883363bc4c8d, scope.project.id=3130efb60669479aae97cf2ec518e76d, user.id=5181c4b31ac240469aff7a0aee35eb7e) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:35.369118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2455fd-f927-4391-b1b8-49997ac28e34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:35.370544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2455fd-f927-4391-b1b8-49997ac28e34 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:35.376781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 647/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 20:12:35 2022] GET /identity/v3/role_assignments?user.id=5181c4b31ac240469aff7a0aee35eb7e&scope.project.id=3130efb60669479aae97cf2ec518e76d&role.id=36331ec562ff42f883fc883363bc4c8d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:37.494478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c8e3d-5aab-40f4-aedb-be41408c3cf4 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:37.495052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c8e3d-5aab-40f4-aedb-be41408c3cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:37.495052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c8e3d-5aab-40f4-aedb-be41408c3cf4 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:37.496364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 641/1288] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:37.501938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3800a-86a2-47c1-9129-38885258d8b8 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:37.502344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3800a-86a2-47c1-9129-38885258d8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:37.502480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3800a-86a2-47c1-9129-38885258d8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:37.521481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-29f3800a-86a2-47c1-9129-38885258d8b8 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:37.522677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-29f3800a-86a2-47c1-9129-38885258d8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:37.553710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:37.596674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 648/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 20:12:37.865523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c8e3d-5aab-40f4-aedb-be41408c3cf4 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:37.965255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c8e3d-5aab-40f4-aedb-be41408c3cf4 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:37.975863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:37.998390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e372b9-356c-4e5c-a652-3cb98c30cc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:37.998996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e372b9-356c-4e5c-a652-3cb98c30cc16 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:37.999268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e372b9-356c-4e5c-a652-3cb98c30cc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:37.999530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e372b9-356c-4e5c-a652-3cb98c30cc16 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:38.000090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e372b9-356c-4e5c-a652-3cb98c30cc16 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:38.001434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e372b9-356c-4e5c-a652-3cb98c30cc16 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-16e372b9-356c-4e5c-a652-3cb98c30cc16 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:38.009786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:38.010778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:38.011753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 642/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:12:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:38.017353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3800a-86a2-47c1-9129-38885258d8b8 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:38.112806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f3800a-86a2-47c1-9129-38885258d8b8 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:38.146156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128e143-eb1b-447e-b16a-ad0be2850d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:38.146687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128e143-eb1b-447e-b16a-ad0be2850d0b None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:38.146968 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128e143-eb1b-447e-b16a-ad0be2850d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:38.147243 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128e143-eb1b-447e-b16a-ad0be2850d0b None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:38.147862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128e143-eb1b-447e-b16a-ad0be2850d0b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:38.148165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128e143-eb1b-447e-b16a-ad0be2850d0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:38.149547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128e143-eb1b-447e-b16a-ad0be2850d0b None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:38.154212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 649/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Oct 5 20:12:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:38.159777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e372b9-356c-4e5c-a652-3cb98c30cc16 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:38.257879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e372b9-356c-4e5c-a652-3cb98c30cc16 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:38.288961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed69dbd-88aa-4b34-a681-be0f44c7a538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:38.289564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed69dbd-88aa-4b34-a681-be0f44c7a538 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:38.289839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed69dbd-88aa-4b34-a681-be0f44c7a538 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:38.290101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed69dbd-88aa-4b34-a681-be0f44c7a538 None admin] PATH_INFO: `/v3/services` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:38.290709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed69dbd-88aa-4b34-a681-be0f44c7a538 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:38.291001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed69dbd-88aa-4b34-a681-be0f44c7a538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:38.292351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed69dbd-88aa-4b34-a681-be0f44c7a538 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:38.297037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 643/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Oct 5 20:12:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:40.385922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee4f39-8edf-4fc4-b418-0341be38eb75 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:40.386496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee4f39-8edf-4fc4-b418-0341be38eb75 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:40.386496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee4f39-8edf-4fc4-b418-0341be38eb75 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:40.388365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 650/1293] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:40.393567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb43c9-b1ca-4838-85d5-1bb6e3f972c5 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:40.393970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb43c9-b1ca-4838-85d5-1bb6e3f972c5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:40.394115 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb43c9-b1ca-4838-85d5-1bb6e3f972c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:40.411594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7fcb43c9-b1ca-4838-85d5-1bb6e3f972c5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:40.412750 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7fcb43c9-b1ca-4838-85d5-1bb6e3f972c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:40.441126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:40.476780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 644/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:12:40.761181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee4f39-8edf-4fc4-b418-0341be38eb75 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:40.800028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:40.871468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee4f39-8edf-4fc4-b418-0341be38eb75 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:40.904271 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d63e90-abdf-47f0-a36c-5d40dd511f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:40.904864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d63e90-abdf-47f0-a36c-5d40dd511f2d None admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:40.905142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d63e90-abdf-47f0-a36c-5d40dd511f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:40.905414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d63e90-abdf-47f0-a36c-5d40dd511f2d None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:40.906093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d63e90-abdf-47f0-a36c-5d40dd511f2d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:40.907441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d63e90-abdf-47f0-a36c-5d40dd511f2d None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:40.921432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 651/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 20:12:40 2022] POST /identity/v3/services => generated 246 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:12:43.080834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7c23d-0d92-4d93-8f1f-05f6d3631974 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:43.081376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7c23d-0d92-4d93-8f1f-05f6d3631974 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:43.081376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7c23d-0d92-4d93-8f1f-05f6d3631974 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:43.082731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 645/1296] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:43.087833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084dc12-eaf5-4911-b337-1c80f338088f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:43.088224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084dc12-eaf5-4911-b337-1c80f338088f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:43.088380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084dc12-eaf5-4911-b337-1c80f338088f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:43.107325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4084dc12-eaf5-4911-b337-1c80f338088f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:43.108504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4084dc12-eaf5-4911-b337-1c80f338088f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:43.178663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 652/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:12:43.448953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7c23d-0d92-4d93-8f1f-05f6d3631974 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:43.544971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7c23d-0d92-4d93-8f1f-05f6d3631974 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:43.584331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cca24c-3649-4bea-a4e9-3bd72cf30664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:43.584958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cca24c-3649-4bea-a4e9-3bd72cf30664 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:43.585235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cca24c-3649-4bea-a4e9-3bd72cf30664 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:43.585606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cca24c-3649-4bea-a4e9-3bd72cf30664 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:43.586186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cca24c-3649-4bea-a4e9-3bd72cf30664 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:43.587551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cca24c-3649-4bea-a4e9-3bd72cf30664 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-e2cca24c-3649-4bea-a4e9-3bd72cf30664 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:43.596764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:43.598198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:43.599428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 646/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:12:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:43.604514 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084dc12-eaf5-4911-b337-1c80f338088f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:43.623630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:43.701870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084dc12-eaf5-4911-b337-1c80f338088f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:43.733875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083549b-a9b2-46da-9b33-fc844580677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:43.734466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083549b-a9b2-46da-9b33-fc844580677e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:43.734740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083549b-a9b2-46da-9b33-fc844580677e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:43.735006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083549b-a9b2-46da-9b33-fc844580677e None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:43.735616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083549b-a9b2-46da-9b33-fc844580677e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:43.735912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083549b-a9b2-46da-9b33-fc844580677e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:43.737246 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083549b-a9b2-46da-9b33-fc844580677e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:43.742139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 653/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Oct 5 20:12:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:12:43.748749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cca24c-3649-4bea-a4e9-3bd72cf30664 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:43.778897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:43.845001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cca24c-3649-4bea-a4e9-3bd72cf30664 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:43.877278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab734a1b-f2e8-4c05-9603-dbbd94b0eaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:43.877909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab734a1b-f2e8-4c05-9603-dbbd94b0eaab None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:43.878188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab734a1b-f2e8-4c05-9603-dbbd94b0eaab None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:43.878460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab734a1b-f2e8-4c05-9603-dbbd94b0eaab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:43.879148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab734a1b-f2e8-4c05-9603-dbbd94b0eaab None admin] RBAC: Adding query filter params (interface=public, service_id=59758c98c74a47d39ee975821c327dae, region_id=RegionOne) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:43.879453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab734a1b-f2e8-4c05-9603-dbbd94b0eaab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:43.880920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab734a1b-f2e8-4c05-9603-dbbd94b0eaab None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:43.885818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 647/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 20:12:43 2022] GET /identity/v3/endpoints?service_id=59758c98c74a47d39ee975821c327dae&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:43.891397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083549b-a9b2-46da-9b33-fc844580677e None admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:43.969445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:43.987399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083549b-a9b2-46da-9b33-fc844580677e None admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:44.018954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf7cc1-a355-4f50-9a02-c36b65d8bd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:44.019542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf7cc1-a355-4f50-9a02-c36b65d8bd3e None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:44.019814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf7cc1-a355-4f50-9a02-c36b65d8bd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:44.020079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf7cc1-a355-4f50-9a02-c36b65d8bd3e None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:44.020664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf7cc1-a355-4f50-9a02-c36b65d8bd3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:44.022040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf7cc1-a355-4f50-9a02-c36b65d8bd3e None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:44.029021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 654/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 20:12:43 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:12:46.146119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a06a4-b0a3-4992-999c-adf47044a50d None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:46.146613 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a06a4-b0a3-4992-999c-adf47044a50d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:46.146613 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a06a4-b0a3-4992-999c-adf47044a50d None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:46.147903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 648/1302] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:12:46.153007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e2a99-c330-4b33-a220-2cb296183057 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:46.153440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e2a99-c330-4b33-a220-2cb296183057 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:46.153572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e2a99-c330-4b33-a220-2cb296183057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:46.171138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a19e2a99-c330-4b33-a220-2cb296183057 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:12:46.172291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a19e2a99-c330-4b33-a220-2cb296183057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:12:46.235626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 655/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:12:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 20:12:46.504571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a06a4-b0a3-4992-999c-adf47044a50d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:46.595599 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:46.602598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a06a4-b0a3-4992-999c-adf47044a50d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:46.635687 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa729b9-6798-460d-bd08-9d0bc2734045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:46.636301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa729b9-6798-460d-bd08-9d0bc2734045 None admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:46.636575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa729b9-6798-460d-bd08-9d0bc2734045 None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:46.636843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa729b9-6798-460d-bd08-9d0bc2734045 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:46.637412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa729b9-6798-460d-bd08-9d0bc2734045 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:46.638788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa729b9-6798-460d-bd08-9d0bc2734045 None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application [None req-baa729b9-6798-460d-bd08-9d0bc2734045 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 20:12:46.647169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 20:12:46.648297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application with Lock( Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self._enter() Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application return self.creator() Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application created_value = creator( Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: ERROR keystone.server.flask.application Oct 05 20:12:46.649382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 649/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 20:12:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 20:12:46.654145 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e2a99-c330-4b33-a220-2cb296183057 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:46.726603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:46.749934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e2a99-c330-4b33-a220-2cb296183057 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:46.781888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913de19-8e9b-4fc6-b827-de161c74d549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:46.782427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913de19-8e9b-4fc6-b827-de161c74d549 None admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:46.782700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913de19-8e9b-4fc6-b827-de161c74d549 None admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:46.782963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913de19-8e9b-4fc6-b827-de161c74d549 None admin] PATH_INFO: `/v3/services` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:46.783571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913de19-8e9b-4fc6-b827-de161c74d549 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:12:46.783864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913de19-8e9b-4fc6-b827-de161c74d549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:46.785189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913de19-8e9b-4fc6-b827-de161c74d549 None admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:46.790103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 656/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Oct 5 20:12:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:12:46.796309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa729b9-6798-460d-bd08-9d0bc2734045 None admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:12:46.893196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa729b9-6798-460d-bd08-9d0bc2734045 None admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:12:46.926081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cf38d-d61b-41c7-89d0-6951ec4df2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:12:46.926724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cf38d-d61b-41c7-89d0-6951ec4df2cf None admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:12:46.927003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cf38d-d61b-41c7-89d0-6951ec4df2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:12:46.927272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cf38d-d61b-41c7-89d0-6951ec4df2cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:12:46.927934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cf38d-d61b-41c7-89d0-6951ec4df2cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:12:46.929283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cf38d-d61b-41c7-89d0-6951ec4df2cf None admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:12:46.948042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 650/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:12:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:12:56.787703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=30 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:12:56.943852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:13:50.472484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8689e41-7fda-4f73-bc6b-fb6894d46318 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:50.472894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8689e41-7fda-4f73-bc6b-fb6894d46318 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:50.473077 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8689e41-7fda-4f73-bc6b-fb6894d46318 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:50.474671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 657/1307] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:13:50.480338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706522d-e939-4106-bfcb-3a26c0443d0a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:50.480762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706522d-e939-4106-bfcb-3a26c0443d0a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:50.480841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706522d-e939-4106-bfcb-3a26c0443d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:50.616333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-4706522d-e939-4106-bfcb-3a26c0443d0a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:13:50.617560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4706522d-e939-4106-bfcb-3a26c0443d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:50.821337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 651/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:13:51.123012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5cb01-0315-44c6-bb96-ba0304fa853a None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:51.123333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5cb01-0315-44c6-bb96-ba0304fa853a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:51.123549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5cb01-0315-44c6-bb96-ba0304fa853a None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:51.125104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 658/1309] 10.222.0.23 () {54 vars in 964 bytes} [Wed Oct 5 20:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:13:51.131982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322953f-9515-495c-be69-058ca9951b4c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:51.132765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322953f-9515-495c-be69-058ca9951b4c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:51.133221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322953f-9515-495c-be69-058ca9951b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:51.226911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-3322953f-9515-495c-be69-058ca9951b4c None None] MFA Rules not processed for user `8f5a88f263014f15a6f59cfdaa2bb6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:13:51.228592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3322953f-9515-495c-be69-058ca9951b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:51.251605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:13:51.320288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 652/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Wed Oct 5 20:13:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:13:51.330125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5cb01-0315-44c6-bb96-ba0304fa853a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:13:51.332293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2da5cb01-0315-44c6-bb96-ba0304fa853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:51.468047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5cb01-0315-44c6-bb96-ba0304fa853a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:13:51.564629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037f466-12c7-477d-a333-e5d0dc774c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:13:51.565653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f466-12c7-477d-a333-e5d0dc774c2d service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:51.565931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f466-12c7-477d-a333-e5d0dc774c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:51.566100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037f466-12c7-477d-a333-e5d0dc774c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:51.567632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e037f466-12c7-477d-a333-e5d0dc774c2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:51.626257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:13:51.684152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037f466-12c7-477d-a333-e5d0dc774c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:13:51.685602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e037f466-12c7-477d-a333-e5d0dc774c2d service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:13:51.705158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 659/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:13:54.177077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8feee-a5f6-4c21-9e3d-d777da75a977 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:54.177559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8feee-a5f6-4c21-9e3d-d777da75a977 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:54.177652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8feee-a5f6-4c21-9e3d-d777da75a977 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:54.178983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 653/1312] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:13:54.184116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606d5d8-ee43-4379-be2a-3661e6439821 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:54.184505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606d5d8-ee43-4379-be2a-3661e6439821 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:54.184663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606d5d8-ee43-4379-be2a-3661e6439821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:54.265259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e606d5d8-ee43-4379-be2a-3661e6439821 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:13:54.266547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e606d5d8-ee43-4379-be2a-3661e6439821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:54.363612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 660/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:13:54.646865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8feee-a5f6-4c21-9e3d-d777da75a977 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:13:54.648375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-55f8feee-a5f6-4c21-9e3d-d777da75a977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:54.748393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8feee-a5f6-4c21-9e3d-d777da75a977 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:13:54.816784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:13:54.819347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be36ee8-60e7-4259-bfd5-1dbb406b5f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:13:54.820173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be36ee8-60e7-4259-bfd5-1dbb406b5f27 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:54.820640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be36ee8-60e7-4259-bfd5-1dbb406b5f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:54.821103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be36ee8-60e7-4259-bfd5-1dbb406b5f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:54.822740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9be36ee8-60e7-4259-bfd5-1dbb406b5f27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:54.929027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be36ee8-60e7-4259-bfd5-1dbb406b5f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:13:54.930422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be36ee8-60e7-4259-bfd5-1dbb406b5f27 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:13:54.948720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 654/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:13:57.381810 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da46e-7cef-43ea-9c5a-aadb6c34fb7c None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:57.382329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da46e-7cef-43ea-9c5a-aadb6c34fb7c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:57.382329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da46e-7cef-43ea-9c5a-aadb6c34fb7c None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:57.383675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 661/1315] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:13:57.389592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575e25d-67f9-49f8-bda1-d035b6a82d6a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:57.390019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575e25d-67f9-49f8-bda1-d035b6a82d6a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:57.390128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575e25d-67f9-49f8-bda1-d035b6a82d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:57.454359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1575e25d-67f9-49f8-bda1-d035b6a82d6a None None] MFA Rules not processed for user `cf6ad7faaf3247d5ba5962434f804797`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:13:57.455554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1575e25d-67f9-49f8-bda1-d035b6a82d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:57.505652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:13:57.548933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 655/1316] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:13:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:13:57.838164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308da46e-7cef-43ea-9c5a-aadb6c34fb7c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:13:57.839666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-308da46e-7cef-43ea-9c5a-aadb6c34fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:57.849337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:13:57.942787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308da46e-7cef-43ea-9c5a-aadb6c34fb7c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:13:58.014806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae8b62-1584-45bd-a6d1-3605a446149b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:13:58.015409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae8b62-1584-45bd-a6d1-3605a446149b service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:13:58.015684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae8b62-1584-45bd-a6d1-3605a446149b service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:13:58.015951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae8b62-1584-45bd-a6d1-3605a446149b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:13:58.017505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-aaae8b62-1584-45bd-a6d1-3605a446149b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:13:58.077872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:13:58.124880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae8b62-1584-45bd-a6d1-3605a446149b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:13:58.126321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae8b62-1584-45bd-a6d1-3605a446149b service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:13:58.143724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 662/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:13:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:00.764175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee39e3a-fc11-49f1-92bd-19eeb51cb0f2 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:00.764742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee39e3a-fc11-49f1-92bd-19eeb51cb0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:00.764742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee39e3a-fc11-49f1-92bd-19eeb51cb0f2 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:00.766079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 656/1318] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:00.771213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552fe0e-e965-4af0-aca3-91b2278b7015 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:00.771632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552fe0e-e965-4af0-aca3-91b2278b7015 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:00.771763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552fe0e-e965-4af0-aca3-91b2278b7015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:00.836686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4552fe0e-e965-4af0-aca3-91b2278b7015 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:00.837903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4552fe0e-e965-4af0-aca3-91b2278b7015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:00.934024 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 663/1319] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:01.214639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee39e3a-fc11-49f1-92bd-19eeb51cb0f2 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:01.216142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fee39e3a-fc11-49f1-92bd-19eeb51cb0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:01.317332 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee39e3a-fc11-49f1-92bd-19eeb51cb0f2 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:01.386662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc8826-575e-446e-bbd9-e627fd79474a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:01.387252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc8826-575e-446e-bbd9-e627fd79474a service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:01.387525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc8826-575e-446e-bbd9-e627fd79474a service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:01.387791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc8826-575e-446e-bbd9-e627fd79474a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:01.389336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-17fc8826-575e-446e-bbd9-e627fd79474a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:01.396721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:01.496100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc8826-575e-446e-bbd9-e627fd79474a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:01.497500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc8826-575e-446e-bbd9-e627fd79474a service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:01.515401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 657/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:04.303743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6671ac-70cb-4c8d-9537-9d5c811eb943 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:04.304266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6671ac-70cb-4c8d-9537-9d5c811eb943 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:04.304266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6671ac-70cb-4c8d-9537-9d5c811eb943 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:04.305591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 664/1321] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:04.310817 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd18222-c75f-454e-ac74-22a1f1421e86 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:04.311259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd18222-c75f-454e-ac74-22a1f1421e86 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:04.311363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd18222-c75f-454e-ac74-22a1f1421e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:04.384496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-dfd18222-c75f-454e-ac74-22a1f1421e86 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:04.385772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-dfd18222-c75f-454e-ac74-22a1f1421e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:04.447355 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:04.490779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 658/1322] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:04.773157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6671ac-70cb-4c8d-9537-9d5c811eb943 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:04.775272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6b6671ac-70cb-4c8d-9537-9d5c811eb943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:04.796114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:04.880045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6671ac-70cb-4c8d-9537-9d5c811eb943 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:04.956772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b21f0-73a2-41f3-8bb6-b3ec2bc8ac5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:04.957374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b21f0-73a2-41f3-8bb6-b3ec2bc8ac5a service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:04.957682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b21f0-73a2-41f3-8bb6-b3ec2bc8ac5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:04.957952 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b21f0-73a2-41f3-8bb6-b3ec2bc8ac5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:04.959501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-eb3b21f0-73a2-41f3-8bb6-b3ec2bc8ac5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:05.030843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:05.068737 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b21f0-73a2-41f3-8bb6-b3ec2bc8ac5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:05.070196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b21f0-73a2-41f3-8bb6-b3ec2bc8ac5a service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:05.088418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 665/1323] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:07.282617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63464ecf-2cf6-4b44-bcca-cba162ba3577 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:07.283135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63464ecf-2cf6-4b44-bcca-cba162ba3577 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:07.283135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63464ecf-2cf6-4b44-bcca-cba162ba3577 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:07.284434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 659/1324] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:07.290482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a24996-8980-4e74-ae66-0e1cf4b2a9f1 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:07.290879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a24996-8980-4e74-ae66-0e1cf4b2a9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:07.291023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a24996-8980-4e74-ae66-0e1cf4b2a9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:07.357215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-48a24996-8980-4e74-ae66-0e1cf4b2a9f1 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:07.358708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-48a24996-8980-4e74-ae66-0e1cf4b2a9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:07.461202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 666/1325] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:07.740479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63464ecf-2cf6-4b44-bcca-cba162ba3577 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:07.742030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-63464ecf-2cf6-4b44-bcca-cba162ba3577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:07.846875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63464ecf-2cf6-4b44-bcca-cba162ba3577 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:07.921336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1328df-9955-4f54-a455-9ee5cf11a556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:07.922008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1328df-9955-4f54-a455-9ee5cf11a556 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:07.922289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1328df-9955-4f54-a455-9ee5cf11a556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:07.922557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1328df-9955-4f54-a455-9ee5cf11a556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:07.924125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fe1328df-9955-4f54-a455-9ee5cf11a556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:07.941652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:08.032517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1328df-9955-4f54-a455-9ee5cf11a556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:08.033892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1328df-9955-4f54-a455-9ee5cf11a556 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:08.052573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 660/1326] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:10.260055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502d4ce-a421-42d0-9175-dd587acc46c4 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:10.260569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502d4ce-a421-42d0-9175-dd587acc46c4 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:10.260675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502d4ce-a421-42d0-9175-dd587acc46c4 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:10.262194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 667/1327] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:10.267590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75b556-d8e8-430e-b72e-372fed607a40 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:10.268025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75b556-d8e8-430e-b72e-372fed607a40 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:10.268137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75b556-d8e8-430e-b72e-372fed607a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:10.336034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ad75b556-d8e8-430e-b72e-372fed607a40 None None] MFA Rules not processed for user `cf6ad7faaf3247d5ba5962434f804797`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:10.337924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ad75b556-d8e8-430e-b72e-372fed607a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:10.404882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:10.441919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 661/1328] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:10.724264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502d4ce-a421-42d0-9175-dd587acc46c4 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:10.725944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6502d4ce-a421-42d0-9175-dd587acc46c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:10.759307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:10.836031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502d4ce-a421-42d0-9175-dd587acc46c4 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:10.912482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eaa74c-2e57-456c-a8f1-6221b854b774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:10.913090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaa74c-2e57-456c-a8f1-6221b854b774 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:10.913364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaa74c-2e57-456c-a8f1-6221b854b774 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:10.913663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaa74c-2e57-456c-a8f1-6221b854b774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:10.915857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-17eaa74c-2e57-456c-a8f1-6221b854b774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:10.994810 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:11.027567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eaa74c-2e57-456c-a8f1-6221b854b774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:11.028985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eaa74c-2e57-456c-a8f1-6221b854b774 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:11.047311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 668/1329] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:13.403836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7ff87-53c3-4a0a-a80a-0487cc21e61b None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:13.404359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7ff87-53c3-4a0a-a80a-0487cc21e61b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:13.404359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7ff87-53c3-4a0a-a80a-0487cc21e61b None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:13.405722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 662/1330] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:13.410940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01788690-bc40-4ece-9551-5be3586fc318 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:13.411348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01788690-bc40-4ece-9551-5be3586fc318 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:13.411488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01788690-bc40-4ece-9551-5be3586fc318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:13.476224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-01788690-bc40-4ece-9551-5be3586fc318 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:13.478043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-01788690-bc40-4ece-9551-5be3586fc318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:13.573258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 669/1331] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:13.854737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7ff87-53c3-4a0a-a80a-0487cc21e61b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:13.856246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-28c7ff87-53c3-4a0a-a80a-0487cc21e61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:13.962811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7ff87-53c3-4a0a-a80a-0487cc21e61b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:14.034061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5ea9d-d893-491b-95cd-874047fe698a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:14.034655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5ea9d-d893-491b-95cd-874047fe698a service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:14.034928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5ea9d-d893-491b-95cd-874047fe698a service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:14.035199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5ea9d-d893-491b-95cd-874047fe698a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:14.036748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-86d5ea9d-d893-491b-95cd-874047fe698a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:14.063003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:14.143261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5ea9d-d893-491b-95cd-874047fe698a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:14.144691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5ea9d-d893-491b-95cd-874047fe698a service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:14.162543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 663/1332] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:16.326616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef25d60-419e-4d5a-8fd3-83ab6f4545fc None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:16.327112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef25d60-419e-4d5a-8fd3-83ab6f4545fc None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:16.327112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef25d60-419e-4d5a-8fd3-83ab6f4545fc None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:16.328464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 670/1333] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:16.333747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8238322-1835-414e-a256-ef30b6bed96a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:16.334190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8238322-1835-414e-a256-ef30b6bed96a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:16.334313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8238322-1835-414e-a256-ef30b6bed96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:16.400586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d8238322-1835-414e-a256-ef30b6bed96a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:16.401826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d8238322-1835-414e-a256-ef30b6bed96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:16.474295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:16.496406 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 664/1334] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:16.777151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef25d60-419e-4d5a-8fd3-83ab6f4545fc None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:16.778709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-bef25d60-419e-4d5a-8fd3-83ab6f4545fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:16.819017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:16.880899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef25d60-419e-4d5a-8fd3-83ab6f4545fc None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:16.953349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d5999-6140-41f7-888b-ddf1348948cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:16.953971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d5999-6140-41f7-888b-ddf1348948cf service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:16.954242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d5999-6140-41f7-888b-ddf1348948cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:16.954504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d5999-6140-41f7-888b-ddf1348948cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:16.956049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-740d5999-6140-41f7-888b-ddf1348948cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:17.037665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:17.063530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d5999-6140-41f7-888b-ddf1348948cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:17.065205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740d5999-6140-41f7-888b-ddf1348948cf service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:17.082681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 671/1335] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:19.506778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395ccbc-221d-48f1-b747-7a507f7a878e None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:19.507323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395ccbc-221d-48f1-b747-7a507f7a878e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:19.507323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395ccbc-221d-48f1-b747-7a507f7a878e None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:19.509530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 665/1336] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:19.514634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeabe2-1696-48d1-946a-a0ca19e08e47 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:19.515050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeabe2-1696-48d1-946a-a0ca19e08e47 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:19.515175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daeabe2-1696-48d1-946a-a0ca19e08e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:19.583651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-0daeabe2-1696-48d1-946a-a0ca19e08e47 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:19.584862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0daeabe2-1696-48d1-946a-a0ca19e08e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:19.679643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 672/1337] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:14:19.975893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5bcd-fa32-4652-9cf1-f06036159005 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:19.976200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5bcd-fa32-4652-9cf1-f06036159005 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:19.976467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd5bcd-fa32-4652-9cf1-f06036159005 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:19.977833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 666/1338] 10.222.0.23 () {54 vars in 964 bytes} [Wed Oct 5 20:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:19.983588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcb08c-72fb-4271-af39-0cecc0d6db37 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:19.983877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcb08c-72fb-4271-af39-0cecc0d6db37 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:19.984147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcb08c-72fb-4271-af39-0cecc0d6db37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:20.037990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:20.060236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-bbfcb08c-72fb-4271-af39-0cecc0d6db37 None None] MFA Rules not processed for user `8f5a88f263014f15a6f59cfdaa2bb6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:20.061464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-bbfcb08c-72fb-4271-af39-0cecc0d6db37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:20.136745 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 673/1339] 10.222.0.23 () {58 vars in 1061 bytes} [Wed Oct 5 20:14:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:20.145724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5bcd-fa32-4652-9cf1-f06036159005 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:20.147290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c6fd5bcd-fa32-4652-9cf1-f06036159005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:20.248868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd5bcd-fa32-4652-9cf1-f06036159005 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:20.321198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1c7d4-3c70-4546-9a46-d78103f37d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:20.321813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1c7d4-3c70-4546-9a46-d78103f37d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:20.322093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1c7d4-3c70-4546-9a46-d78103f37d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:20.322359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1c7d4-3c70-4546-9a46-d78103f37d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:20.323898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1fd1c7d4-3c70-4546-9a46-d78103f37d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:20.358113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:20.428432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1c7d4-3c70-4546-9a46-d78103f37d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:20.429804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1c7d4-3c70-4546-9a46-d78103f37d55 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:20.448106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 667/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:25.067041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4460728-174b-4950-af35-36aa5972880c None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:25.067589 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4460728-174b-4950-af35-36aa5972880c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:25.067589 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4460728-174b-4950-af35-36aa5972880c None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:25.068820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 674/1341] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:25.074693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2435fbc-b5a3-4bb8-97fb-9f9e4d7cac57 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:25.075146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2435fbc-b5a3-4bb8-97fb-9f9e4d7cac57 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:25.075374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2435fbc-b5a3-4bb8-97fb-9f9e4d7cac57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:25.141382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a2435fbc-b5a3-4bb8-97fb-9f9e4d7cac57 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:25.142540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a2435fbc-b5a3-4bb8-97fb-9f9e4d7cac57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:25.225706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:25.237008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 668/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:25.518437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4460728-174b-4950-af35-36aa5972880c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:25.519985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b4460728-174b-4950-af35-36aa5972880c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:25.626660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4460728-174b-4950-af35-36aa5972880c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:25.666518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:25.703770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8968087-8d46-4827-b70a-1f7996798a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:25.704373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968087-8d46-4827-b70a-1f7996798a09 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:25.704647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968087-8d46-4827-b70a-1f7996798a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:25.704910 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8968087-8d46-4827-b70a-1f7996798a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:25.706500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e8968087-8d46-4827-b70a-1f7996798a09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:25.818953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968087-8d46-4827-b70a-1f7996798a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:25.820299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8968087-8d46-4827-b70a-1f7996798a09 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:25.839834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 675/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:28.603664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70872ab-eead-4089-90f6-ba5461614b1d None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:28.604184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70872ab-eead-4089-90f6-ba5461614b1d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:28.604276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70872ab-eead-4089-90f6-ba5461614b1d None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:28.605666 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 669/1344] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:28.611129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58de93e-11ba-47f4-aca0-2acab2c7dba8 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:28.611538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58de93e-11ba-47f4-aca0-2acab2c7dba8 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:28.611753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58de93e-11ba-47f4-aca0-2acab2c7dba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:28.678367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c58de93e-11ba-47f4-aca0-2acab2c7dba8 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:28.679568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c58de93e-11ba-47f4-aca0-2acab2c7dba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:28.698239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:28.781931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 676/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:29.073438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70872ab-eead-4089-90f6-ba5461614b1d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:29.075017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f70872ab-eead-4089-90f6-ba5461614b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:29.184813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70872ab-eead-4089-90f6-ba5461614b1d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:29.257091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb57c0-ddb2-4c77-925f-2c3ca2ea41a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:29.257720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb57c0-ddb2-4c77-925f-2c3ca2ea41a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:29.257993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb57c0-ddb2-4c77-925f-2c3ca2ea41a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:29.258256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb57c0-ddb2-4c77-925f-2c3ca2ea41a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:29.259802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a4bb57c0-ddb2-4c77-925f-2c3ca2ea41a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:29.303058 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:29.368556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb57c0-ddb2-4c77-925f-2c3ca2ea41a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:29.370036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb57c0-ddb2-4c77-925f-2c3ca2ea41a4 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:29.388055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 670/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:33.487123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168fbd6-75b1-4938-8bd0-b58a62f2409a None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:33.487585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168fbd6-75b1-4938-8bd0-b58a62f2409a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:33.487675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168fbd6-75b1-4938-8bd0-b58a62f2409a None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:33.489003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 677/1347] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:33.494900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed46a5-4faf-40bc-8da8-a4205f348960 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:33.495321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed46a5-4faf-40bc-8da8-a4205f348960 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:33.495453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed46a5-4faf-40bc-8da8-a4205f348960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:33.563792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-15ed46a5-4faf-40bc-8da8-a4205f348960 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:33.565279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-15ed46a5-4faf-40bc-8da8-a4205f348960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:33.662521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:33.665158 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 671/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:33.951660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168fbd6-75b1-4938-8bd0-b58a62f2409a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:33.953247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5168fbd6-75b1-4938-8bd0-b58a62f2409a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:34.059617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168fbd6-75b1-4938-8bd0-b58a62f2409a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:34.107196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:34.132576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255fb89-34b9-4459-82a4-516bdae596a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:34.133175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255fb89-34b9-4459-82a4-516bdae596a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:34.133449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255fb89-34b9-4459-82a4-516bdae596a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:34.133755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255fb89-34b9-4459-82a4-516bdae596a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:34.135300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f255fb89-34b9-4459-82a4-516bdae596a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:34.247722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255fb89-34b9-4459-82a4-516bdae596a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:34.249272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f255fb89-34b9-4459-82a4-516bdae596a9 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:34.267508 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 678/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:39.107081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839741db-eec5-4a72-89e5-c59b7b59d152 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:39.107520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839741db-eec5-4a72-89e5-c59b7b59d152 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:39.107623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839741db-eec5-4a72-89e5-c59b7b59d152 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:39.109251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 672/1350] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:39.115288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9c9d7-f6dc-46b8-aec7-60bf12191bdf None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:39.115699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9c9d7-f6dc-46b8-aec7-60bf12191bdf None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:39.115874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9c9d7-f6dc-46b8-aec7-60bf12191bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:39.220261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-14b9c9d7-f6dc-46b8-aec7-60bf12191bdf None None] MFA Rules not processed for user `5181c4b31ac240469aff7a0aee35eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:39.222065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-14b9c9d7-f6dc-46b8-aec7-60bf12191bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:39.245118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:39.374046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 679/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:14:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:43.583050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc71f9f-44cb-4256-8e80-b2038a73b854 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:43.583575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc71f9f-44cb-4256-8e80-b2038a73b854 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:43.583575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc71f9f-44cb-4256-8e80-b2038a73b854 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:43.585083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 673/1352] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:43.590801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd096a-57f6-4eac-8160-30fb6da99214 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:43.591578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd096a-57f6-4eac-8160-30fb6da99214 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:43.592148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd096a-57f6-4eac-8160-30fb6da99214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:43.676524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-61bd096a-57f6-4eac-8160-30fb6da99214 None None] MFA Rules not processed for user `5181c4b31ac240469aff7a0aee35eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:43.678114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-61bd096a-57f6-4eac-8160-30fb6da99214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:43.762201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:43.802943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 680/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:14:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:45.240879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dfa9f-bf0e-4942-891e-2ea5a41a81bc None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:45.241414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dfa9f-bf0e-4942-891e-2ea5a41a81bc None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:45.241414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48dfa9f-bf0e-4942-891e-2ea5a41a81bc None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:45.242832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 674/1354] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:45.247938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d2a46-a41b-428e-9ae7-3febeffca21f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:45.248378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d2a46-a41b-428e-9ae7-3febeffca21f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:45.248641 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d2a46-a41b-428e-9ae7-3febeffca21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:45.327347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a80d2a46-a41b-428e-9ae7-3febeffca21f None None] MFA Rules not processed for user `5181c4b31ac240469aff7a0aee35eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:45.328525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a80d2a46-a41b-428e-9ae7-3febeffca21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:45.433878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 681/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:14:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:49.053208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c11d8-4721-4b02-8d88-aa543a57b316 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:49.055087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c11d8-4721-4b02-8d88-aa543a57b316 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:49.055431 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c11d8-4721-4b02-8d88-aa543a57b316 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:49.056884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 675/1356] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:49.062615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c9de5-d601-456d-b228-f403f94646b5 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:49.063435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c9de5-d601-456d-b228-f403f94646b5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:49.063720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c9de5-d601-456d-b228-f403f94646b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:49.103937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:49.137897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ea4c9de5-d601-456d-b228-f403f94646b5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:49.139004 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ea4c9de5-d601-456d-b228-f403f94646b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:49.243962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 682/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:49.544249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b62cf1-c4fe-41c4-b193-b109d4a16e75 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:49.545086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b62cf1-c4fe-41c4-b193-b109d4a16e75 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:49.545712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b62cf1-c4fe-41c4-b193-b109d4a16e75 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:49.547559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 676/1358] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 20:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:49.557872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c14161-f4e9-4ec7-b2fb-746538a7491c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:49.558422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c14161-f4e9-4ec7-b2fb-746538a7491c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:49.558903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c14161-f4e9-4ec7-b2fb-746538a7491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:49.644838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f2c14161-f4e9-4ec7-b2fb-746538a7491c None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:49.646048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f2c14161-f4e9-4ec7-b2fb-746538a7491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:49.700075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:49.774343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 683/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 20:14:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:49.782190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b62cf1-c4fe-41c4-b193-b109d4a16e75 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:49.783688 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-65b62cf1-c4fe-41c4-b193-b109d4a16e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:49.945446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b62cf1-c4fe-41c4-b193-b109d4a16e75 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:50.020474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:50.061920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80396354-c75a-4c39-ac90-5f88ce4aa423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:50.062687 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80396354-c75a-4c39-ac90-5f88ce4aa423 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:50.062890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80396354-c75a-4c39-ac90-5f88ce4aa423 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:50.063157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80396354-c75a-4c39-ac90-5f88ce4aa423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:50.065080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-80396354-c75a-4c39-ac90-5f88ce4aa423 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:50.160003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80396354-c75a-4c39-ac90-5f88ce4aa423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:50.161653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80396354-c75a-4c39-ac90-5f88ce4aa423 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:50.181373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 677/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:52.566804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d30a4-a6f0-4b97-b33f-1e7c233d3244 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:52.567428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d30a4-a6f0-4b97-b33f-1e7c233d3244 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:52.567428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d30a4-a6f0-4b97-b33f-1e7c233d3244 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:52.568680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 684/1361] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:52.574157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0a24c-f342-4826-8e83-114852217d11 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:52.574606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0a24c-f342-4826-8e83-114852217d11 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:52.574708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0a24c-f342-4826-8e83-114852217d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:52.664218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-61a0a24c-f342-4826-8e83-114852217d11 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:52.665746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-61a0a24c-f342-4826-8e83-114852217d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:52.711768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:52.776900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 678/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:14:53.062720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d30a4-a6f0-4b97-b33f-1e7c233d3244 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:53.064222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c38d30a4-a6f0-4b97-b33f-1e7c233d3244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:53.223771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d30a4-a6f0-4b97-b33f-1e7c233d3244 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:53.236732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:53.365886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796ef34-c518-4729-96e6-229e98adf2aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:53.367009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796ef34-c518-4729-96e6-229e98adf2aa service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:53.367275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796ef34-c518-4729-96e6-229e98adf2aa service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:53.367551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796ef34-c518-4729-96e6-229e98adf2aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:53.369122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4796ef34-c518-4729-96e6-229e98adf2aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:53.467873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796ef34-c518-4729-96e6-229e98adf2aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:53.469653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796ef34-c518-4729-96e6-229e98adf2aa service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:53.488979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 685/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:55.974157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f279109-0387-467e-aa8d-281169337348 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:55.974675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f279109-0387-467e-aa8d-281169337348 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:55.974675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f279109-0387-467e-aa8d-281169337348 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:55.976938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 679/1364] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:55.982427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6795b4-498f-4176-8b78-8206c94f45df None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:55.982864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6795b4-498f-4176-8b78-8206c94f45df None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:55.982964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6795b4-498f-4176-8b78-8206c94f45df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:55.999636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:56.056937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-cd6795b4-498f-4176-8b78-8206c94f45df None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:56.058296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-cd6795b4-498f-4176-8b78-8206c94f45df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:56.166842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 686/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:56.456061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0bf89-5167-4ce0-b85b-5d416f3fab3d None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:56.456349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0bf89-5167-4ce0-b85b-5d416f3fab3d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:56.456627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0bf89-5167-4ce0-b85b-5d416f3fab3d None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:56.457975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 680/1366] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 20:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:56.467638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0c488-8ecd-4a4d-95d8-104360cd3292 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:56.467954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0c488-8ecd-4a4d-95d8-104360cd3292 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:56.468196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0c488-8ecd-4a4d-95d8-104360cd3292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:56.552372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c1f0c488-8ecd-4a4d-95d8-104360cd3292 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:56.554352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c1f0c488-8ecd-4a4d-95d8-104360cd3292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:56.593677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:56.681123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 687/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 20:14:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:56.688858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0bf89-5167-4ce0-b85b-5d416f3fab3d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:56.690662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-68e0bf89-5167-4ce0-b85b-5d416f3fab3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:56.844489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0bf89-5167-4ce0-b85b-5d416f3fab3d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:56.869940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:56.971273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe6811-baca-4802-bee1-6e609cbc7224 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:56.971871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6811-baca-4802-bee1-6e609cbc7224 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:56.972158 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6811-baca-4802-bee1-6e609cbc7224 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:56.972416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe6811-baca-4802-bee1-6e609cbc7224 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:56.974240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-dbbe6811-baca-4802-bee1-6e609cbc7224 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:57.070234 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe6811-baca-4802-bee1-6e609cbc7224 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:57.072230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe6811-baca-4802-bee1-6e609cbc7224 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:57.094183 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 681/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:57.271519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f23fb-facb-476d-8f98-669b3dfe14b6 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:57.272110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f23fb-facb-476d-8f98-669b3dfe14b6 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:57.272110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f23fb-facb-476d-8f98-669b3dfe14b6 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:57.273570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 688/1369] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:57.279261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e255e0-eeac-4a33-bb5c-8b23fb35be1d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:57.279556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e255e0-eeac-4a33-bb5c-8b23fb35be1d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:57.279794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e255e0-eeac-4a33-bb5c-8b23fb35be1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:57.315923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:57.360002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-10e255e0-eeac-4a33-bb5c-8b23fb35be1d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:57.361142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-10e255e0-eeac-4a33-bb5c-8b23fb35be1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:57.474094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 682/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:57.813654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee2a68-32b3-4baf-82d7-d9c3e8af1ef1 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:57.813654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee2a68-32b3-4baf-82d7-d9c3e8af1ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:57.814002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee2a68-32b3-4baf-82d7-d9c3e8af1ef1 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:57.815514 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 689/1371] 10.222.0.23 () {54 vars in 963 bytes} [Wed Oct 5 20:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:57.823101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68e167-9f8f-485b-b0ae-802a4fe24190 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:57.823101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68e167-9f8f-485b-b0ae-802a4fe24190 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:57.823101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68e167-9f8f-485b-b0ae-802a4fe24190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:57.911973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7c68e167-9f8f-485b-b0ae-802a4fe24190 None None] MFA Rules not processed for user `a6e8bc66986d48aa9dcbc19258ff6e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:57.913739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7c68e167-9f8f-485b-b0ae-802a4fe24190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:57.964292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:57.997747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 683/1372] 10.222.0.23 () {58 vars in 1060 bytes} [Wed Oct 5 20:14:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:58.005277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee2a68-32b3-4baf-82d7-d9c3e8af1ef1 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:58.006940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ccee2a68-32b3-4baf-82d7-d9c3e8af1ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:58.120459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee2a68-32b3-4baf-82d7-d9c3e8af1ef1 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:14:58.165155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:58.205573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72174f-7b31-4aea-82e3-d6c0fb0ba4d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:14:58.206286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72174f-7b31-4aea-82e3-d6c0fb0ba4d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:58.206491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72174f-7b31-4aea-82e3-d6c0fb0ba4d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:58.206898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72174f-7b31-4aea-82e3-d6c0fb0ba4d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:58.208486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1a72174f-7b31-4aea-82e3-d6c0fb0ba4d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:58.338651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72174f-7b31-4aea-82e3-d6c0fb0ba4d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:14:58.340106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72174f-7b31-4aea-82e3-d6c0fb0ba4d0 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:14:58.360769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 690/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:14:59.312050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce401c-6820-4167-b151-2f9fbfd31c26 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:59.312584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce401c-6820-4167-b151-2f9fbfd31c26 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:59.312584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce401c-6820-4167-b151-2f9fbfd31c26 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:59.313902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 684/1374] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:14:59.320115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a75e-b153-42d2-8ca4-2aab49967079 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:14:59.320641 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a75e-b153-42d2-8ca4-2aab49967079 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:14:59.321136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a75e-b153-42d2-8ca4-2aab49967079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:14:59.388434 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-5853a75e-b153-42d2-8ca4-2aab49967079 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:14:59.389655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5853a75e-b153-42d2-8ca4-2aab49967079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:59.410895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:14:59.488532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 691/1375] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:14:59.783742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce401c-6820-4167-b151-2f9fbfd31c26 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:14:59.785250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-efce401c-6820-4167-b151-2f9fbfd31c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:14:59.950127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce401c-6820-4167-b151-2f9fbfd31c26 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:00.007128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:00.072900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cce9e3-2a5d-4520-881c-3c519998586d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:00.073526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cce9e3-2a5d-4520-881c-3c519998586d service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:00.073804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cce9e3-2a5d-4520-881c-3c519998586d service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:00.074071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cce9e3-2a5d-4520-881c-3c519998586d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:00.075678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f1cce9e3-2a5d-4520-881c-3c519998586d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:00.168816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cce9e3-2a5d-4520-881c-3c519998586d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:00.170436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cce9e3-2a5d-4520-881c-3c519998586d service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:00.188512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 685/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:02.411891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0860bd-5793-436f-b5f1-5ad0d861ecf4 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:02.412388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0860bd-5793-436f-b5f1-5ad0d861ecf4 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:02.412388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0860bd-5793-436f-b5f1-5ad0d861ecf4 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:02.414049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 692/1377] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:02.419211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c9c1f-cf59-437c-a443-bd639a34e0d6 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:02.419637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c9c1f-cf59-437c-a443-bd639a34e0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:02.419802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c9c1f-cf59-437c-a443-bd639a34e0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:02.488560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-097c9c1f-cf59-437c-a443-bd639a34e0d6 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:02.489781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-097c9c1f-cf59-437c-a443-bd639a34e0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:02.497806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:02.592670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 686/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:02.885226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0860bd-5793-436f-b5f1-5ad0d861ecf4 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:02.886888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3d0860bd-5793-436f-b5f1-5ad0d861ecf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:03.041442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0860bd-5793-436f-b5f1-5ad0d861ecf4 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:03.045483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:03.162754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d3086e-bb48-45ed-9b25-debc817e848c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:03.163517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3086e-bb48-45ed-9b25-debc817e848c service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:03.163895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3086e-bb48-45ed-9b25-debc817e848c service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:03.164262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3086e-bb48-45ed-9b25-debc817e848c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:03.165962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-39d3086e-bb48-45ed-9b25-debc817e848c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:03.259641 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3086e-bb48-45ed-9b25-debc817e848c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:03.261709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d3086e-bb48-45ed-9b25-debc817e848c service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:03.280847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 693/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:05.506196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6abba-357e-4555-9a03-790b385447b0 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:05.506769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6abba-357e-4555-9a03-790b385447b0 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:05.506769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6abba-357e-4555-9a03-790b385447b0 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:05.509934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 687/1380] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:15:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:05.513265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e44101-91e3-47e6-a89f-bc3373e67b1a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:05.513679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e44101-91e3-47e6-a89f-bc3373e67b1a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:05.513861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e44101-91e3-47e6-a89f-bc3373e67b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:05.525588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:05.586634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-73e44101-91e3-47e6-a89f-bc3373e67b1a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:05.587839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-73e44101-91e3-47e6-a89f-bc3373e67b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:05.692637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 694/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:05.987044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6abba-357e-4555-9a03-790b385447b0 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:05.988720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-18d6abba-357e-4555-9a03-790b385447b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:06.127611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:06.139707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6abba-357e-4555-9a03-790b385447b0 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:06.258716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a1d91-0d2b-4bc4-912e-41e0850c99bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:06.259318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1d91-0d2b-4bc4-912e-41e0850c99bc service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:06.259590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1d91-0d2b-4bc4-912e-41e0850c99bc service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:06.259854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a1d91-0d2b-4bc4-912e-41e0850c99bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:06.261421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a04a1d91-0d2b-4bc4-912e-41e0850c99bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:06.349883 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a1d91-0d2b-4bc4-912e-41e0850c99bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:06.351503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a1d91-0d2b-4bc4-912e-41e0850c99bc service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:06.361434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:06.370603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 688/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:08.587457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415599e6-3f0f-416f-8b14-19d7fe950a95 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:08.588157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415599e6-3f0f-416f-8b14-19d7fe950a95 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:08.588157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415599e6-3f0f-416f-8b14-19d7fe950a95 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:08.589336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 695/1383] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:08.595374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801ac40-1bee-415f-a006-12241b1bf217 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:08.596008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801ac40-1bee-415f-a006-12241b1bf217 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:08.596409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801ac40-1bee-415f-a006-12241b1bf217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:08.665639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8801ac40-1bee-415f-a006-12241b1bf217 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:08.666855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8801ac40-1bee-415f-a006-12241b1bf217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:08.766574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 689/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:09.137098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415599e6-3f0f-416f-8b14-19d7fe950a95 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:09.138610 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-415599e6-3f0f-416f-8b14-19d7fe950a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:09.229990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:09.294102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415599e6-3f0f-416f-8b14-19d7fe950a95 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:09.412831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7fe0f-354b-496d-82b0-dec413e77a9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:09.413364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7fe0f-354b-496d-82b0-dec413e77a9f service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:09.413690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7fe0f-354b-496d-82b0-dec413e77a9f service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:09.413956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7fe0f-354b-496d-82b0-dec413e77a9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:09.415548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-65f7fe0f-354b-496d-82b0-dec413e77a9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:09.467726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:09.516706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7fe0f-354b-496d-82b0-dec413e77a9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:09.518389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7fe0f-354b-496d-82b0-dec413e77a9f service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:09.536982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 696/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:11.831268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fc89e-e8f6-44a6-9ff8-37585e80a16a None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:11.833263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fc89e-e8f6-44a6-9ff8-37585e80a16a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:11.833263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fc89e-e8f6-44a6-9ff8-37585e80a16a None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:11.833970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 690/1386] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:11.841207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f083d04-026f-4dec-969c-e4ba2c0d15b9 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:11.841797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f083d04-026f-4dec-969c-e4ba2c0d15b9 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:11.841797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f083d04-026f-4dec-969c-e4ba2c0d15b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:11.912088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7f083d04-026f-4dec-969c-e4ba2c0d15b9 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:11.913866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7f083d04-026f-4dec-969c-e4ba2c0d15b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:12.011366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 697/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:12.301264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fc89e-e8f6-44a6-9ff8-37585e80a16a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:12.302849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f57fc89e-e8f6-44a6-9ff8-37585e80a16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:12.379748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:12.456369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fc89e-e8f6-44a6-9ff8-37585e80a16a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:12.570030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3835a2-c1e9-44fb-8053-e5a46cf0aaee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:12.570634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3835a2-c1e9-44fb-8053-e5a46cf0aaee service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:12.570915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3835a2-c1e9-44fb-8053-e5a46cf0aaee service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:12.571181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3835a2-c1e9-44fb-8053-e5a46cf0aaee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:12.572757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fa3835a2-c1e9-44fb-8053-e5a46cf0aaee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:12.610672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:12.663553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3835a2-c1e9-44fb-8053-e5a46cf0aaee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:12.665399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3835a2-c1e9-44fb-8053-e5a46cf0aaee service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:12.684302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 691/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:13.690791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ee842-28f9-4a5a-864d-5043fe39daf8 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:13.691352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ee842-28f9-4a5a-864d-5043fe39daf8 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:13.691352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ee842-28f9-4a5a-864d-5043fe39daf8 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:13.692729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 698/1389] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:15:13.698629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b8192-e721-483c-bc2d-d64ed367b342 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:13.699036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b8192-e721-483c-bc2d-d64ed367b342 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:13.699172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b8192-e721-483c-bc2d-d64ed367b342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:13.764605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-801b8192-e721-483c-bc2d-d64ed367b342 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:13.765853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-801b8192-e721-483c-bc2d-d64ed367b342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:13.865988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 692/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:15:14.168438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f720b-bc31-4acd-a1b9-85a6d13c49c9 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:14.168722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f720b-bc31-4acd-a1b9-85a6d13c49c9 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:14.168996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f720b-bc31-4acd-a1b9-85a6d13c49c9 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:14.170442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 699/1391] 10.222.0.23 () {54 vars in 962 bytes} [Wed Oct 5 20:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:14.175825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f06f51-ce06-4b88-845c-3ce721f5bf67 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:14.176178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f06f51-ce06-4b88-845c-3ce721f5bf67 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:14.176513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f06f51-ce06-4b88-845c-3ce721f5bf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:14.192410 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:14.367764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-69f06f51-ce06-4b88-845c-3ce721f5bf67 None None] MFA Rules not processed for user `47bafea2367e4c7992354bb2c23e7d57`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:14.368961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-69f06f51-ce06-4b88-845c-3ce721f5bf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:14.445867 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 693/1392] 10.222.0.23 () {58 vars in 1059 bytes} [Wed Oct 5 20:15:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:14.454886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f720b-bc31-4acd-a1b9-85a6d13c49c9 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:14.456351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1d0f720b-bc31-4acd-a1b9-85a6d13c49c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:14.487528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:14.565184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f720b-bc31-4acd-a1b9-85a6d13c49c9 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:14.638850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2941da-6ea1-438a-b470-3b184ba91b10 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bafea2367e4c7992354bb2c23e7d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:14.639450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2941da-6ea1-438a-b470-3b184ba91b10 service glance] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:14.639722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2941da-6ea1-438a-b470-3b184ba91b10 service glance] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:14.639984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2941da-6ea1-438a-b470-3b184ba91b10 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:14.641535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fe2941da-6ea1-438a-b470-3b184ba91b10 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:14.718958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:14.751006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2941da-6ea1-438a-b470-3b184ba91b10 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:14.752453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2941da-6ea1-438a-b470-3b184ba91b10 service glance] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:14.773631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 700/1393] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:14.875863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8ca20-704c-4830-bb3c-8b89e60fbce9 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:14.876409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8ca20-704c-4830-bb3c-8b89e60fbce9 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:14.876409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8ca20-704c-4830-bb3c-8b89e60fbce9 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:14.877759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 694/1394] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:14.883521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407a4c-dd16-4ada-bc78-27e20a5cae91 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:14.883808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407a4c-dd16-4ada-bc78-27e20a5cae91 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:14.884083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58407a4c-dd16-4ada-bc78-27e20a5cae91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:14.956307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-58407a4c-dd16-4ada-bc78-27e20a5cae91 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:14.957560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-58407a4c-dd16-4ada-bc78-27e20a5cae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:15.063767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 701/1395] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:15.366802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8ca20-704c-4830-bb3c-8b89e60fbce9 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:15.368373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b6d8ca20-704c-4830-bb3c-8b89e60fbce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:15.471521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:15.522722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8ca20-704c-4830-bb3c-8b89e60fbce9 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:15.535330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81221d-96ce-49b9-a585-e6ecfd0e0aa9 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:15.535617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81221d-96ce-49b9-a585-e6ecfd0e0aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:15.535889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81221d-96ce-49b9-a585-e6ecfd0e0aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:15.583063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:15.605472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-fc81221d-96ce-49b9-a585-e6ecfd0e0aa9 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:15.606644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fc81221d-96ce-49b9-a585-e6ecfd0e0aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:15.637672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:15.638280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:15.638552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:15.638819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:15.640385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:15.697669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 702/1396] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:15:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:15:15.713747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:15.736738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:15.738322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:15.745813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a638a34-e78d-45d7-afea-2982a15015e2 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:15.746296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a638a34-e78d-45d7-afea-2982a15015e2 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:15.746401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a638a34-e78d-45d7-afea-2982a15015e2 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:15.747864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 703/1397] 10.222.0.23 () {54 vars in 943 bytes} [Wed Oct 5 20:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:15:15.754022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3845f8-5c09-4395-a369-91b1cc3e54f6 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:15.754304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3845f8-5c09-4395-a369-91b1cc3e54f6 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:15.754580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3845f8-5c09-4395-a369-91b1cc3e54f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:15.757789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 695/1398] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:15.840623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-2d3845f8-5c09-4395-a369-91b1cc3e54f6 None None] MFA Rules not processed for user `f8dec1dbdcd94316aad25c93c313a5a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:15.841909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2d3845f8-5c09-4395-a369-91b1cc3e54f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:15.915588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:15.966623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 704/1399] 10.222.0.23 () {58 vars in 1040 bytes} [Wed Oct 5 20:15:15 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:15:15.976369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:15.977621 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:16.064147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ecfe-770b-4806-996a-3abf69d3f574 service nova] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:16.169306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:16.170907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd4d23-e5bf-4425-94e9-b9b845a06194 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:16.171784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd4d23-e5bf-4425-94e9-b9b845a06194 service swift] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:16.172071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd4d23-e5bf-4425-94e9-b9b845a06194 service swift] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:16.172651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd4d23-e5bf-4425-94e9-b9b845a06194 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:16.174767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-14dd4d23-e5bf-4425-94e9-b9b845a06194 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:16.257664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd4d23-e5bf-4425-94e9-b9b845a06194 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:16.259245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd4d23-e5bf-4425-94e9-b9b845a06194 service swift] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:16.267339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 696/1400] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:15:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:15:17.943146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663ac3-b450-4ceb-8a40-39ff5ba2b3e5 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:17.943702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663ac3-b450-4ceb-8a40-39ff5ba2b3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:17.943702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663ac3-b450-4ceb-8a40-39ff5ba2b3e5 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:17.945043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 705/1401] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:17.950242 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba389c-11c8-473f-bad4-cce8058c6f41 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:17.950644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba389c-11c8-473f-bad4-cce8058c6f41 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:17.950824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba389c-11c8-473f-bad4-cce8058c6f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:18.017850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5aba389c-11c8-473f-bad4-cce8058c6f41 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:18.018970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5aba389c-11c8-473f-bad4-cce8058c6f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:18.113463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:18.121855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 697/1402] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:15:18.412101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6663ac3-b450-4ceb-8a40-39ff5ba2b3e5 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:18.413614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e6663ac3-b450-4ceb-8a40-39ff5ba2b3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:18.574336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6663ac3-b450-4ceb-8a40-39ff5ba2b3e5 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:18.618779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:18.699691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f4ed0-46ed-407e-a6ac-0951179e691b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:18.700210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4ed0-46ed-407e-a6ac-0951179e691b service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:18.700602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4ed0-46ed-407e-a6ac-0951179e691b service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:18.700869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4ed0-46ed-407e-a6ac-0951179e691b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:18.702496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d70f4ed0-46ed-407e-a6ac-0951179e691b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:18.803744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f4ed0-46ed-407e-a6ac-0951179e691b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:18.805334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f4ed0-46ed-407e-a6ac-0951179e691b service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:18.823940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 706/1403] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:21.045818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419e88-e675-400a-836c-196dbb3b9c0f None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:21.046326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419e88-e675-400a-836c-196dbb3b9c0f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:21.046441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419e88-e675-400a-836c-196dbb3b9c0f None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:21.047853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 698/1404] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:21.053553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7bc76-ab07-44ae-8c52-cdeda2fbfdfb None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:21.053982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7bc76-ab07-44ae-8c52-cdeda2fbfdfb None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:21.054092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7bc76-ab07-44ae-8c52-cdeda2fbfdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:21.117410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:21.126584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-93b7bc76-ab07-44ae-8c52-cdeda2fbfdfb None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:21.127806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-93b7bc76-ab07-44ae-8c52-cdeda2fbfdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:21.237338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 707/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:21.529484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419e88-e675-400a-836c-196dbb3b9c0f None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:21.531021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d3419e88-e675-400a-836c-196dbb3b9c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:21.753029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419e88-e675-400a-836c-196dbb3b9c0f None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:21.830743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:21.889391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e2b47-9599-4d84-bd4a-1424efe96af3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:21.890152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e2b47-9599-4d84-bd4a-1424efe96af3 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:21.890499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e2b47-9599-4d84-bd4a-1424efe96af3 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:21.890840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e2b47-9599-4d84-bd4a-1424efe96af3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:21.892500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f13e2b47-9599-4d84-bd4a-1424efe96af3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:21.993882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e2b47-9599-4d84-bd4a-1424efe96af3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:21.995626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e2b47-9599-4d84-bd4a-1424efe96af3 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:22.014622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 699/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:24.242401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3236764-fad4-4e7b-badc-ffed2e04133b None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:24.242980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3236764-fad4-4e7b-badc-ffed2e04133b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:24.242980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3236764-fad4-4e7b-badc-ffed2e04133b None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:24.244327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 708/1407] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:24.249688 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab7007-ea1f-489f-b9c4-ea9381299a1b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:24.250113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab7007-ea1f-489f-b9c4-ea9381299a1b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:24.250299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab7007-ea1f-489f-b9c4-ea9381299a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:24.324869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-2dab7007-ea1f-489f-b9c4-ea9381299a1b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:24.326163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2dab7007-ea1f-489f-b9c4-ea9381299a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:24.359507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:24.433965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 700/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:15:24.725282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3236764-fad4-4e7b-badc-ffed2e04133b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:24.727082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b3236764-fad4-4e7b-badc-ffed2e04133b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:24.866200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:24.903231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3236764-fad4-4e7b-badc-ffed2e04133b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:25.031083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35444636-67de-4b9c-9444-4c36f46af558 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:25.031596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35444636-67de-4b9c-9444-4c36f46af558 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:25.031868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35444636-67de-4b9c-9444-4c36f46af558 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:25.032127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35444636-67de-4b9c-9444-4c36f46af558 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:25.033737 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-35444636-67de-4b9c-9444-4c36f46af558 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:25.130157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:25.136169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35444636-67de-4b9c-9444-4c36f46af558 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:25.137902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35444636-67de-4b9c-9444-4c36f46af558 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:25.158802 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 709/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:34.432496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=30 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:35.156352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=7 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:38.841083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a90bdd-b354-4a71-8614-688f2afb9782 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:38.841673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a90bdd-b354-4a71-8614-688f2afb9782 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:38.841673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a90bdd-b354-4a71-8614-688f2afb9782 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:38.842997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 701/1410] 10.222.0.23 () {54 vars in 918 bytes} [Wed Oct 5 20:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:38.848085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ac702-89f3-43c8-a4ab-f1777f9ae97a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:38.848494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ac702-89f3-43c8-a4ab-f1777f9ae97a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:38.848624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ac702-89f3-43c8-a4ab-f1777f9ae97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:38.923974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8c1ac702-89f3-43c8-a4ab-f1777f9ae97a None None] MFA Rules not processed for user `5181c4b31ac240469aff7a0aee35eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:38.925164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8c1ac702-89f3-43c8-a4ab-f1777f9ae97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:39.030803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 710/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Oct 5 20:15:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:39.045210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d92a62-7e59-4f64-b094-3b8e9905352d None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:39.045547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d92a62-7e59-4f64-b094-3b8e9905352d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:39.045828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d92a62-7e59-4f64-b094-3b8e9905352d None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:39.047156 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 702/1412] 10.222.0.23 () {54 vars in 918 bytes} [Wed Oct 5 20:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:39.051762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2641a5-9076-4c6b-b75b-e0718450ac14 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:39.052642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2641a5-9076-4c6b-b75b-e0718450ac14 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:39.052956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2641a5-9076-4c6b-b75b-e0718450ac14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:39.126505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:39.129490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4b2641a5-9076-4c6b-b75b-e0718450ac14 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:39.131884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4b2641a5-9076-4c6b-b75b-e0718450ac14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:39.249135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 711/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Oct 5 20:15:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:41.820916 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14441e66-c61c-478b-889b-f71f5b048087 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:41.821490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14441e66-c61c-478b-889b-f71f5b048087 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:41.821490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14441e66-c61c-478b-889b-f71f5b048087 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:41.822842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:41.828002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda389b4-5a01-4ef6-84c5-cb2f310193d9 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:41.828415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda389b4-5a01-4ef6-84c5-cb2f310193d9 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:41.828528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda389b4-5a01-4ef6-84c5-cb2f310193d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:41.896196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-dda389b4-5a01-4ef6-84c5-cb2f310193d9 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:41.897376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-dda389b4-5a01-4ef6-84c5-cb2f310193d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:41.950971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:41.992176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:42.278709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14441e66-c61c-478b-889b-f71f5b048087 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:42.280269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-14441e66-c61c-478b-889b-f71f5b048087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:42.423192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14441e66-c61c-478b-889b-f71f5b048087 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:42.494534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:42.533584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b59236-acd3-4c6f-8748-78cc67d936de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:42.534181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59236-acd3-4c6f-8748-78cc67d936de service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:42.534455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59236-acd3-4c6f-8748-78cc67d936de service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:42.534723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59236-acd3-4c6f-8748-78cc67d936de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:42.536291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-29b59236-acd3-4c6f-8748-78cc67d936de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:42.623076 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59236-acd3-4c6f-8748-78cc67d936de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:42.624646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59236-acd3-4c6f-8748-78cc67d936de service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:42.642827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 704/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:43.777299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aad320-eae5-44c8-a3ce-97921c408103 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:43.777893 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aad320-eae5-44c8-a3ce-97921c408103 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:43.777893 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aad320-eae5-44c8-a3ce-97921c408103 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:43.779218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:43.785268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b399769-8b6e-48ef-aeb7-48531015fd59 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:43.785721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b399769-8b6e-48ef-aeb7-48531015fd59 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:43.785855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b399769-8b6e-48ef-aeb7-48531015fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:43.861584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-2b399769-8b6e-48ef-aeb7-48531015fd59 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:43.862810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2b399769-8b6e-48ef-aeb7-48531015fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:43.889427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:43.958868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:44.108393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aad320-eae5-44c8-a3ce-97921c408103 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:44.109896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-26aad320-eae5-44c8-a3ce-97921c408103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:44.253765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aad320-eae5-44c8-a3ce-97921c408103 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:44.294015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:44.365623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca642f55-f97e-4392-a9f7-9e8804e1db01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:44.366163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca642f55-f97e-4392-a9f7-9e8804e1db01 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:44.366438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca642f55-f97e-4392-a9f7-9e8804e1db01 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:44.366708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca642f55-f97e-4392-a9f7-9e8804e1db01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:44.368284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ca642f55-f97e-4392-a9f7-9e8804e1db01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:44.457253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca642f55-f97e-4392-a9f7-9e8804e1db01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:44.459419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca642f55-f97e-4392-a9f7-9e8804e1db01 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:44.476776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 714/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:45.608823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d369f-3e38-434e-8f1c-04a498090968 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:45.609388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d369f-3e38-434e-8f1c-04a498090968 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:45.609388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d369f-3e38-434e-8f1c-04a498090968 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:45.610722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:45.616911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd2131-3af3-4ad0-8d51-106ac553f6eb None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:45.617319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd2131-3af3-4ad0-8d51-106ac553f6eb None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:45.617471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd2131-3af3-4ad0-8d51-106ac553f6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:45.671998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:45.694074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e4bd2131-3af3-4ad0-8d51-106ac553f6eb None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:45.695263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e4bd2131-3af3-4ad0-8d51-106ac553f6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:45.791280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:45.943451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d369f-3e38-434e-8f1c-04a498090968 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:45.944983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-338d369f-3e38-434e-8f1c-04a498090968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:46.089389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d369f-3e38-434e-8f1c-04a498090968 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:46.101685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:46.199083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50aae36-bda9-4f90-b274-1c7101a499f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:46.199712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50aae36-bda9-4f90-b274-1c7101a499f9 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:46.199986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50aae36-bda9-4f90-b274-1c7101a499f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:46.200253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50aae36-bda9-4f90-b274-1c7101a499f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:46.201845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b50aae36-bda9-4f90-b274-1c7101a499f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:46.290691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50aae36-bda9-4f90-b274-1c7101a499f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:46.292269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50aae36-bda9-4f90-b274-1c7101a499f9 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:46.310116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 707/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:47.411617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8219dc-288c-48c1-b855-b54f87bad354 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:47.412149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8219dc-288c-48c1-b855-b54f87bad354 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:47.412149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8219dc-288c-48c1-b855-b54f87bad354 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:47.413403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:47.419421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dd14a-c41d-40dc-8cea-0c20e3daf795 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:47.419825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dd14a-c41d-40dc-8cea-0c20e3daf795 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:47.419971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dd14a-c41d-40dc-8cea-0c20e3daf795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:47.439700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:47.494955 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-fd8dd14a-c41d-40dc-8cea-0c20e3daf795 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:47.496155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fd8dd14a-c41d-40dc-8cea-0c20e3daf795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:47.591977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:47.740230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8219dc-288c-48c1-b855-b54f87bad354 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:47.741675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4d8219dc-288c-48c1-b855-b54f87bad354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:47.852050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:47.887914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8219dc-288c-48c1-b855-b54f87bad354 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:47.999575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d355618-24fd-4d1d-99f7-36b9e2d27759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:48.000105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355618-24fd-4d1d-99f7-36b9e2d27759 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:48.000383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355618-24fd-4d1d-99f7-36b9e2d27759 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:48.000649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355618-24fd-4d1d-99f7-36b9e2d27759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:48.002248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4d355618-24fd-4d1d-99f7-36b9e2d27759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:48.078785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:48.093156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d355618-24fd-4d1d-99f7-36b9e2d27759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:48.094787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d355618-24fd-4d1d-99f7-36b9e2d27759 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:48.112305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 717/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:49.210372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aec8c4-64ec-4351-b05b-792b64d83404 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:49.210830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aec8c4-64ec-4351-b05b-792b64d83404 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:49.210920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aec8c4-64ec-4351-b05b-792b64d83404 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:49.212231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:49.218538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33692a6b-a7df-4f16-ad56-2583129e438b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:49.218969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33692a6b-a7df-4f16-ad56-2583129e438b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:49.219086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33692a6b-a7df-4f16-ad56-2583129e438b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:49.294123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-33692a6b-a7df-4f16-ad56-2583129e438b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:49.295334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-33692a6b-a7df-4f16-ad56-2583129e438b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:49.392980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:49.543552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aec8c4-64ec-4351-b05b-792b64d83404 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:49.545055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a1aec8c4-64ec-4351-b05b-792b64d83404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:49.629361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:49.690771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aec8c4-64ec-4351-b05b-792b64d83404 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:49.801087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a490d1-5560-4b13-8af7-41a54ed3c8b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:49.801635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a490d1-5560-4b13-8af7-41a54ed3c8b2 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:49.802451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a490d1-5560-4b13-8af7-41a54ed3c8b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:49.802717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a490d1-5560-4b13-8af7-41a54ed3c8b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:49.804283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c6a490d1-5560-4b13-8af7-41a54ed3c8b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:49.852298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:49.892872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a490d1-5560-4b13-8af7-41a54ed3c8b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:49.894494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a490d1-5560-4b13-8af7-41a54ed3c8b2 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:49.912975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 710/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:50.966046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfda73-c490-4d7c-a5b4-41c7db765964 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:50.966601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfda73-c490-4d7c-a5b4-41c7db765964 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:50.966601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfda73-c490-4d7c-a5b4-41c7db765964 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:50.967927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:50.974706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8cd76-b9a0-4928-b35b-974f3e19d0ca None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:50.975118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8cd76-b9a0-4928-b35b-974f3e19d0ca None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:50.975251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8cd76-b9a0-4928-b35b-974f3e19d0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:51.050287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-0ea8cd76-b9a0-4928-b35b-974f3e19d0ca None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:51.051412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0ea8cd76-b9a0-4928-b35b-974f3e19d0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:51.144905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:51.293885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfda73-c490-4d7c-a5b4-41c7db765964 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:51.295374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-27dfda73-c490-4d7c-a5b4-41c7db765964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:51.344285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:51.441105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfda73-c490-4d7c-a5b4-41c7db765964 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:51.550817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272c094-193c-4853-972b-8bc34fee42a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:51.551411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272c094-193c-4853-972b-8bc34fee42a1 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:51.551707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272c094-193c-4853-972b-8bc34fee42a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:51.551969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272c094-193c-4853-972b-8bc34fee42a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:51.553540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8272c094-193c-4853-972b-8bc34fee42a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:51.560243 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:51.642745 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272c094-193c-4853-972b-8bc34fee42a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:51.644245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272c094-193c-4853-972b-8bc34fee42a1 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:51.662749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 720/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:52.721151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493278d-1e24-459d-8f6c-ca80c8519427 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:52.721711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493278d-1e24-459d-8f6c-ca80c8519427 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:52.721711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493278d-1e24-459d-8f6c-ca80c8519427 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:52.723055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:52.729170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0bd06-48d8-4960-b2a4-7c48549027e6 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:52.729590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0bd06-48d8-4960-b2a4-7c48549027e6 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:52.729750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0bd06-48d8-4960-b2a4-7c48549027e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:52.805001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-dbb0bd06-48d8-4960-b2a4-7c48549027e6 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:52.806134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-dbb0bd06-48d8-4960-b2a4-7c48549027e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:52.875964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:52.904711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:53.054238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493278d-1e24-459d-8f6c-ca80c8519427 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:53.055786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f493278d-1e24-459d-8f6c-ca80c8519427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:53.073038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:53.198654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493278d-1e24-459d-8f6c-ca80c8519427 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:53.282702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:53.309327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74458b04-9066-4c99-9af0-3c1c4fe17e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:53.309867 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458b04-9066-4c99-9af0-3c1c4fe17e02 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:53.310138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458b04-9066-4c99-9af0-3c1c4fe17e02 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:53.310401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74458b04-9066-4c99-9af0-3c1c4fe17e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:53.311948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-74458b04-9066-4c99-9af0-3c1c4fe17e02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:53.398743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74458b04-9066-4c99-9af0-3c1c4fe17e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:53.400315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74458b04-9066-4c99-9af0-3c1c4fe17e02 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:53.418213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 713/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:54.475435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c665f4-8165-41e3-894b-ffc1247d9a2f None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:54.475971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c665f4-8165-41e3-894b-ffc1247d9a2f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:54.475971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c665f4-8165-41e3-894b-ffc1247d9a2f None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:54.477242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:54.483540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116ae21-3fde-4b8e-8226-3bb71f034fe6 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:54.483926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116ae21-3fde-4b8e-8226-3bb71f034fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:54.484087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116ae21-3fde-4b8e-8226-3bb71f034fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:54.559698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a116ae21-3fde-4b8e-8226-3bb71f034fe6 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:54.560907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a116ae21-3fde-4b8e-8226-3bb71f034fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:54.602015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:54.655678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:54.804157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c665f4-8165-41e3-894b-ffc1247d9a2f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:54.805683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b5c665f4-8165-41e3-894b-ffc1247d9a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:54.953984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c665f4-8165-41e3-894b-ffc1247d9a2f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:55.007836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:55.066443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac3aa5-aa1e-4bba-9db5-51100221c900 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:55.066973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac3aa5-aa1e-4bba-9db5-51100221c900 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:55.067765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac3aa5-aa1e-4bba-9db5-51100221c900 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:55.068030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac3aa5-aa1e-4bba-9db5-51100221c900 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:55.069615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-daac3aa5-aa1e-4bba-9db5-51100221c900 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:55.156987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac3aa5-aa1e-4bba-9db5-51100221c900 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:55.158585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac3aa5-aa1e-4bba-9db5-51100221c900 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:55.176245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 723/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:56.219445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab40f7-cdbd-4490-9ea6-d21bc609543a None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:56.219914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab40f7-cdbd-4490-9ea6-d21bc609543a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:56.220017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab40f7-cdbd-4490-9ea6-d21bc609543a None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:56.221313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:56.228094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1f9b5-e7ec-49c6-9e2b-6471e54c0afb None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:56.228485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1f9b5-e7ec-49c6-9e2b-6471e54c0afb None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:56.228633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1f9b5-e7ec-49c6-9e2b-6471e54c0afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:56.306398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-03d1f9b5-e7ec-49c6-9e2b-6471e54c0afb None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:56.307603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-03d1f9b5-e7ec-49c6-9e2b-6471e54c0afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:56.309664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:56.403560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:56.553046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab40f7-cdbd-4490-9ea6-d21bc609543a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:56.554487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-49ab40f7-cdbd-4490-9ea6-d21bc609543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:56.697086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab40f7-cdbd-4490-9ea6-d21bc609543a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:56.724380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:56.807838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59059ae-7a0f-4ab0-aea0-08152afc69dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:56.808359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59059ae-7a0f-4ab0-aea0-08152afc69dd service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:56.808632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59059ae-7a0f-4ab0-aea0-08152afc69dd service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:56.808897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59059ae-7a0f-4ab0-aea0-08152afc69dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:56.810500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f59059ae-7a0f-4ab0-aea0-08152afc69dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:56.899722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59059ae-7a0f-4ab0-aea0-08152afc69dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:56.901324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59059ae-7a0f-4ab0-aea0-08152afc69dd service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:56.919445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 716/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:15:58.970066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078f47a-2210-4cbc-9e94-b2f12c20d885 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:58.970684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078f47a-2210-4cbc-9e94-b2f12c20d885 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:58.970684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078f47a-2210-4cbc-9e94-b2f12c20d885 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:58.971971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:15:58.977225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906edaf-7652-4805-b6d8-913f8ca99b80 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:58.977746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906edaf-7652-4805-b6d8-913f8ca99b80 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:58.977852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906edaf-7652-4805-b6d8-913f8ca99b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:59.019338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:59.044863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8906edaf-7652-4805-b6d8-913f8ca99b80 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:15:59.046092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8906edaf-7652-4805-b6d8-913f8ca99b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:59.140473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:15:59.428054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078f47a-2210-4cbc-9e94-b2f12c20d885 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:15:59.429614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6078f47a-2210-4cbc-9e94-b2f12c20d885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:59.531924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078f47a-2210-4cbc-9e94-b2f12c20d885 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:15:59.563545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:15:59.607945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8845ff-d49d-4298-b9f3-14bf8c2c3f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:15:59.608544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8845ff-d49d-4298-b9f3-14bf8c2c3f62 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:15:59.608815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8845ff-d49d-4298-b9f3-14bf8c2c3f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:15:59.609075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8845ff-d49d-4298-b9f3-14bf8c2c3f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:15:59.610660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fd8845ff-d49d-4298-b9f3-14bf8c2c3f62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:15:59.718560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8845ff-d49d-4298-b9f3-14bf8c2c3f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:15:59.719907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8845ff-d49d-4298-b9f3-14bf8c2c3f62 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:15:59.737984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 726/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:02.001699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c3be4-e542-4b2b-a5ee-0150e80b3407 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:02.002280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c3be4-e542-4b2b-a5ee-0150e80b3407 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:02.002280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c3be4-e542-4b2b-a5ee-0150e80b3407 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:02.003559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:02.008821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31a6c9-738f-47b0-ad04-fff027a70a86 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:02.009252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31a6c9-738f-47b0-ad04-fff027a70a86 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:02.009378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a31a6c9-738f-47b0-ad04-fff027a70a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:02.074994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8a31a6c9-738f-47b0-ad04-fff027a70a86 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:02.076186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8a31a6c9-738f-47b0-ad04-fff027a70a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:02.089118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:02.170964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:02.451959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c3be4-e542-4b2b-a5ee-0150e80b3407 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:02.453462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f40c3be4-e542-4b2b-a5ee-0150e80b3407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:02.554595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c3be4-e542-4b2b-a5ee-0150e80b3407 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:02.567494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:02.629040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb41967-90e9-4869-81fb-b3342eeebe69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:02.629699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41967-90e9-4869-81fb-b3342eeebe69 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:02.629976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41967-90e9-4869-81fb-b3342eeebe69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:02.630250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb41967-90e9-4869-81fb-b3342eeebe69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:02.631856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4bb41967-90e9-4869-81fb-b3342eeebe69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:02.739160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb41967-90e9-4869-81fb-b3342eeebe69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:02.740518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb41967-90e9-4869-81fb-b3342eeebe69 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:02.757895 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 719/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:05.220339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50661568-c817-4050-b2ae-cf1ba0caed06 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:05.220856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50661568-c817-4050-b2ae-cf1ba0caed06 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:05.220856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50661568-c817-4050-b2ae-cf1ba0caed06 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:05.222189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:05.228004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45901e1-3a56-4e37-87df-7b0545bbf6a1 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:05.228441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45901e1-3a56-4e37-87df-7b0545bbf6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:05.228535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45901e1-3a56-4e37-87df-7b0545bbf6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:05.274915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:05.292753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d45901e1-3a56-4e37-87df-7b0545bbf6a1 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:05.294032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d45901e1-3a56-4e37-87df-7b0545bbf6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:05.390295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:05.675920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50661568-c817-4050-b2ae-cf1ba0caed06 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:05.677465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-50661568-c817-4050-b2ae-cf1ba0caed06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:05.781348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50661568-c817-4050-b2ae-cf1ba0caed06 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:05.821240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:05.856132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab3730-aa2b-4ae1-bdf7-569187bf8959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:05.856735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab3730-aa2b-4ae1-bdf7-569187bf8959 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:05.857010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab3730-aa2b-4ae1-bdf7-569187bf8959 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:05.857278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab3730-aa2b-4ae1-bdf7-569187bf8959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:05.859044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-bdab3730-aa2b-4ae1-bdf7-569187bf8959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:05.969437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab3730-aa2b-4ae1-bdf7-569187bf8959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:05.970822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab3730-aa2b-4ae1-bdf7-569187bf8959 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:05.988921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 729/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:08.239336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9d68a-c870-49ed-a66c-2b7abd2e514d None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:08.239857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9d68a-c870-49ed-a66c-2b7abd2e514d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:08.239980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9d68a-c870-49ed-a66c-2b7abd2e514d None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:08.241893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:08.247275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14fadb-256d-449a-8a77-320fd239d41f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:08.247668 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14fadb-256d-449a-8a77-320fd239d41f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:08.247823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14fadb-256d-449a-8a77-320fd239d41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:08.313958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-dc14fadb-256d-449a-8a77-320fd239d41f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:08.315150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-dc14fadb-256d-449a-8a77-320fd239d41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:08.335752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:08.414639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:08.702454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9d68a-c870-49ed-a66c-2b7abd2e514d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:08.704003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-eca9d68a-c870-49ed-a66c-2b7abd2e514d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:08.805961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9d68a-c870-49ed-a66c-2b7abd2e514d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:08.826140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:08.879094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873999c-7c67-49a1-bdc3-ec66b043939e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:08.879736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873999c-7c67-49a1-bdc3-ec66b043939e service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:08.880014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873999c-7c67-49a1-bdc3-ec66b043939e service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:08.880282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873999c-7c67-49a1-bdc3-ec66b043939e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:08.881973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b873999c-7c67-49a1-bdc3-ec66b043939e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:08.990012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873999c-7c67-49a1-bdc3-ec66b043939e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:08.991945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873999c-7c67-49a1-bdc3-ec66b043939e service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:09.009375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 722/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:11.220424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad0de8-f36c-4126-a718-b5b4ec956a44 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:11.220957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad0de8-f36c-4126-a718-b5b4ec956a44 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:11.220957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad0de8-f36c-4126-a718-b5b4ec956a44 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:11.222364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:11.228307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078208a-594c-4e4d-a26a-8adb48889061 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:11.228730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078208a-594c-4e4d-a26a-8adb48889061 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:11.228853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078208a-594c-4e4d-a26a-8adb48889061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:11.292070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:11.295339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c078208a-594c-4e4d-a26a-8adb48889061 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:11.296737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c078208a-594c-4e4d-a26a-8adb48889061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:11.389911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:11.681582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad0de8-f36c-4126-a718-b5b4ec956a44 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:11.683094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-09ad0de8-f36c-4126-a718-b5b4ec956a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:11.785923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad0de8-f36c-4126-a718-b5b4ec956a44 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:11.835942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:11.860680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719fbe0-d1b7-4f6a-983a-08af3db89182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:11.861413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719fbe0-d1b7-4f6a-983a-08af3db89182 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:11.862142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719fbe0-d1b7-4f6a-983a-08af3db89182 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:11.862418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719fbe0-d1b7-4f6a-983a-08af3db89182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:11.864044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1719fbe0-d1b7-4f6a-983a-08af3db89182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:11.975403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719fbe0-d1b7-4f6a-983a-08af3db89182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:11.976748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719fbe0-d1b7-4f6a-983a-08af3db89182 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:11.994276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 732/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:14.180828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a07d48-1e14-4464-8576-38e6e76fa280 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:14.181337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a07d48-1e14-4464-8576-38e6e76fa280 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:14.181337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a07d48-1e14-4464-8576-38e6e76fa280 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:14.182729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:14.188260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8c9b6-c410-4328-9dcb-6f0a6d9252f4 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:14.188709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8c9b6-c410-4328-9dcb-6f0a6d9252f4 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:14.188816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8c9b6-c410-4328-9dcb-6f0a6d9252f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:14.255734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-cfb8c9b6-c410-4328-9dcb-6f0a6d9252f4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:14.256959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-cfb8c9b6-c410-4328-9dcb-6f0a6d9252f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:14.285034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:14.352499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:14.631981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a07d48-1e14-4464-8576-38e6e76fa280 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:14.633481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c0a07d48-1e14-4464-8576-38e6e76fa280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:14.734145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a07d48-1e14-4464-8576-38e6e76fa280 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:14.763268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:14.807648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0887f-b071-4479-9125-b76f7380da20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:14.808246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0887f-b071-4479-9125-b76f7380da20 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:14.808520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0887f-b071-4479-9125-b76f7380da20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:14.809380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0887f-b071-4479-9125-b76f7380da20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:14.810990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-99c0887f-b071-4479-9125-b76f7380da20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:14.920456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0887f-b071-4479-9125-b76f7380da20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:14.922084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0887f-b071-4479-9125-b76f7380da20 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:14.939557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 725/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:17.138551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3c8cd-27e0-4126-be98-ad514870e3b3 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:17.139117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3c8cd-27e0-4126-be98-ad514870e3b3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:17.139117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3c8cd-27e0-4126-be98-ad514870e3b3 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:17.140432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:17.146668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07db88-0bca-4809-b59e-5c7456917625 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:17.147138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07db88-0bca-4809-b59e-5c7456917625 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:17.147216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07db88-0bca-4809-b59e-5c7456917625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:17.214906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-aa07db88-0bca-4809-b59e-5c7456917625 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:17.216124 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-aa07db88-0bca-4809-b59e-5c7456917625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:17.224368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:17.311267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:17.595309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3c8cd-27e0-4126-be98-ad514870e3b3 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:17.596824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a3d3c8cd-27e0-4126-be98-ad514870e3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:17.699601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3c8cd-27e0-4126-be98-ad514870e3b3 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:17.754768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:17.773362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a8b53-d29a-4fce-a5c6-80c3137ee3d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:17.773991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a8b53-d29a-4fce-a5c6-80c3137ee3d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:17.774269 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a8b53-d29a-4fce-a5c6-80c3137ee3d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:17.774534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a8b53-d29a-4fce-a5c6-80c3137ee3d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:17.776091 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3b0a8b53-d29a-4fce-a5c6-80c3137ee3d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:17.884097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a8b53-d29a-4fce-a5c6-80c3137ee3d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:17.885557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a8b53-d29a-4fce-a5c6-80c3137ee3d3 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:17.904118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 735/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:20.092178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4db67c-2561-44bc-9092-94893c274b4d None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:20.092738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4db67c-2561-44bc-9092-94893c274b4d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:20.092738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4db67c-2561-44bc-9092-94893c274b4d None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:20.094105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:20.099901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1b553-8249-48e5-b79e-c10368862bae None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:20.100306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1b553-8249-48e5-b79e-c10368862bae None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:20.100442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1b553-8249-48e5-b79e-c10368862bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:20.164393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f9b1b553-8249-48e5-b79e-c10368862bae None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:20.165504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f9b1b553-8249-48e5-b79e-c10368862bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:20.203134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:20.260982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:16:20.543345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4db67c-2561-44bc-9092-94893c274b4d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:20.544878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ea4db67c-2561-44bc-9092-94893c274b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:20.647409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4db67c-2561-44bc-9092-94893c274b4d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:20.683581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:20.718561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb8646-c56f-44bc-b1a5-baf95e4342f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:20.719154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb8646-c56f-44bc-b1a5-baf95e4342f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:20.719424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb8646-c56f-44bc-b1a5-baf95e4342f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:20.719685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb8646-c56f-44bc-b1a5-baf95e4342f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:20.721242 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-abeb8646-c56f-44bc-b1a5-baf95e4342f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:20.829087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb8646-c56f-44bc-b1a5-baf95e4342f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:20.830553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb8646-c56f-44bc-b1a5-baf95e4342f3 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:20.849798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 728/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:23.024229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fcdfe-6fb9-4598-91d4-a3b41a6827de None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:23.024788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fcdfe-6fb9-4598-91d4-a3b41a6827de None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:23.024788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fcdfe-6fb9-4598-91d4-a3b41a6827de None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:23.026133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:23.031417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db511c-b326-42e1-8404-54b3b9a55dfd None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:23.031836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db511c-b326-42e1-8404-54b3b9a55dfd None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:23.031966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db511c-b326-42e1-8404-54b3b9a55dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:23.096663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-43db511c-b326-42e1-8404-54b3b9a55dfd None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:23.097962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-43db511c-b326-42e1-8404-54b3b9a55dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:23.115773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:23.194377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:23.480014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fcdfe-6fb9-4598-91d4-a3b41a6827de None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:23.481563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0d5fcdfe-6fb9-4598-91d4-a3b41a6827de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:23.585144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fcdfe-6fb9-4598-91d4-a3b41a6827de None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:23.651386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:23.658327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312965f-53ca-4220-aaba-53b221a82fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:23.658929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312965f-53ca-4220-aaba-53b221a82fdc service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:23.659204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312965f-53ca-4220-aaba-53b221a82fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:23.659472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312965f-53ca-4220-aaba-53b221a82fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:23.661091 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c312965f-53ca-4220-aaba-53b221a82fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:23.768718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312965f-53ca-4220-aaba-53b221a82fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:23.770089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312965f-53ca-4220-aaba-53b221a82fdc service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:23.788858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 738/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:25.979976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ecf4a-67e3-419b-9e3d-724976273f77 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:25.980536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ecf4a-67e3-419b-9e3d-724976273f77 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:25.980536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ecf4a-67e3-419b-9e3d-724976273f77 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:25.981936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:25.987270 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e156c63-a4b6-47ba-b259-5324cfb35672 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:25.987664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e156c63-a4b6-47ba-b259-5324cfb35672 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:25.987836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e156c63-a4b6-47ba-b259-5324cfb35672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:26.169189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4e156c63-a4b6-47ba-b259-5324cfb35672 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:26.170325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4e156c63-a4b6-47ba-b259-5324cfb35672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:26.218448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:26.266169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:26.547846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ecf4a-67e3-419b-9e3d-724976273f77 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:26.549372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-362ecf4a-67e3-419b-9e3d-724976273f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:26.652049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ecf4a-67e3-419b-9e3d-724976273f77 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:26.698759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:26.725071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3394b-14a7-4456-bab9-b5c00b2ae239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:26.725701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3394b-14a7-4456-bab9-b5c00b2ae239 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:26.725975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3394b-14a7-4456-bab9-b5c00b2ae239 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:26.726238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3394b-14a7-4456-bab9-b5c00b2ae239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:26.727795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-33b3394b-14a7-4456-bab9-b5c00b2ae239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:26.834354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3394b-14a7-4456-bab9-b5c00b2ae239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:26.835710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3394b-14a7-4456-bab9-b5c00b2ae239 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:26.853655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 731/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:36.264656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=22 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:36.851392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=66 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:46.664295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea46fe-a814-46a4-ab93-32c85d171398 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:46.664853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea46fe-a814-46a4-ab93-32c85d171398 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:46.664853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea46fe-a814-46a4-ab93-32c85d171398 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:46.666245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 740/1471] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:46.671508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081a579-f5cb-47d1-804a-a53f0e8bb5d4 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:46.671928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081a579-f5cb-47d1-804a-a53f0e8bb5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:46.672024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081a579-f5cb-47d1-804a-a53f0e8bb5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:46.736644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f081a579-f5cb-47d1-804a-a53f0e8bb5d4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:46.737855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f081a579-f5cb-47d1-804a-a53f0e8bb5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:46.830889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 732/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:47.124026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaad58-ec2e-4b8c-97da-e687664a980f None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:47.124311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaad58-ec2e-4b8c-97da-e687664a980f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:47.124629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaad58-ec2e-4b8c-97da-e687664a980f None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:47.126208 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 741/1473] 10.222.0.23 () {54 vars in 962 bytes} [Wed Oct 5 20:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:47.131784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230aa28-188d-4367-9999-85da0dc4755f None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:47.132139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230aa28-188d-4367-9999-85da0dc4755f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:47.132477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230aa28-188d-4367-9999-85da0dc4755f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:47.214046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6230aa28-188d-4367-9999-85da0dc4755f None None] MFA Rules not processed for user `47bafea2367e4c7992354bb2c23e7d57`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:47.215253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6230aa28-188d-4367-9999-85da0dc4755f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:47.235872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:47.293907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 733/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Wed Oct 5 20:16:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:47.302268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaaad58-ec2e-4b8c-97da-e687664a980f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:47.303752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3eaaad58-ec2e-4b8c-97da-e687664a980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:47.406015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaaad58-ec2e-4b8c-97da-e687664a980f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:47.478020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6c1be-c2de-4b3f-b714-c4ca8ef8c7a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bafea2367e4c7992354bb2c23e7d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:47.478622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6c1be-c2de-4b3f-b714-c4ca8ef8c7a9 service glance] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:47.478894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6c1be-c2de-4b3f-b714-c4ca8ef8c7a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:47.479161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6c1be-c2de-4b3f-b714-c4ca8ef8c7a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:47.480703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3cc6c1be-c2de-4b3f-b714-c4ca8ef8c7a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:47.533609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:47.590914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6c1be-c2de-4b3f-b714-c4ca8ef8c7a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:47.592337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6c1be-c2de-4b3f-b714-c4ca8ef8c7a9 service glance] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:47.610374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 742/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:49.860678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ee73d-df58-4c55-ab4e-6a966b94705e None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:49.861216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ee73d-df58-4c55-ab4e-6a966b94705e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:49.861216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ee73d-df58-4c55-ab4e-6a966b94705e None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:49.862623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 734/1476] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:49.868047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d7f8-6308-4661-bbce-aafbce5fb1e7 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:49.868504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d7f8-6308-4661-bbce-aafbce5fb1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:49.868599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d7f8-6308-4661-bbce-aafbce5fb1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:49.933814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4224d7f8-6308-4661-bbce-aafbce5fb1e7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:49.934997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4224d7f8-6308-4661-bbce-aafbce5fb1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:50.029418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 743/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:16:50.306778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ee73d-df58-4c55-ab4e-6a966b94705e None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:50.308285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-973ee73d-df58-4c55-ab4e-6a966b94705e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:50.432606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ee73d-df58-4c55-ab4e-6a966b94705e None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:50.481551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:50.525407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77ad1f-812a-4344-9bdb-0e7f84bbfa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:50.526033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77ad1f-812a-4344-9bdb-0e7f84bbfa20 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:50.526307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77ad1f-812a-4344-9bdb-0e7f84bbfa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:50.526575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77ad1f-812a-4344-9bdb-0e7f84bbfa20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:50.527203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77ad1f-812a-4344-9bdb-0e7f84bbfa20 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:50.528913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING py.warnings [None req-5b77ad1f-812a-4344-9bdb-0e7f84bbfa20 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:16:50.528913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: warnings.warn(msg) Oct 05 20:16:50.528913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: Oct 05 20:16:50.530084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77ad1f-812a-4344-9bdb-0e7f84bbfa20 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:50.549147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 735/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 20:16:50 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:16:52.729106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f08f6-6d2b-4d77-a0d4-72cf00cac58b None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:52.729675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f08f6-6d2b-4d77-a0d4-72cf00cac58b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:52.729675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f08f6-6d2b-4d77-a0d4-72cf00cac58b None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:52.731026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 744/1479] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:52.736369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5551f82-f42e-42d1-813b-eed8e28a5a5d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:52.736767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5551f82-f42e-42d1-813b-eed8e28a5a5d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:52.736917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5551f82-f42e-42d1-813b-eed8e28a5a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:52.802145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a5551f82-f42e-42d1-813b-eed8e28a5a5d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:52.803267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a5551f82-f42e-42d1-813b-eed8e28a5a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:52.899561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 736/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:53.198284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f08f6-6d2b-4d77-a0d4-72cf00cac58b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:53.199785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3b9f08f6-6d2b-4d77-a0d4-72cf00cac58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:53.210144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:53.351553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f08f6-6d2b-4d77-a0d4-72cf00cac58b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:53.430104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:53.463077 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563aac12-138d-4f5b-8b42-11b5de7d8722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:53.463610 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563aac12-138d-4f5b-8b42-11b5de7d8722 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:53.463885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563aac12-138d-4f5b-8b42-11b5de7d8722 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:53.464152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563aac12-138d-4f5b-8b42-11b5de7d8722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:53.465739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-563aac12-138d-4f5b-8b42-11b5de7d8722 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:53.553740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563aac12-138d-4f5b-8b42-11b5de7d8722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:53.555329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563aac12-138d-4f5b-8b42-11b5de7d8722 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:53.574121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 745/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:55.861943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed914b-cb3f-40db-b31c-6317d9352e83 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:55.862488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed914b-cb3f-40db-b31c-6317d9352e83 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:55.862488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed914b-cb3f-40db-b31c-6317d9352e83 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:55.864358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 737/1482] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:55.869710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031a41-ddbe-46d5-b253-a1402c5ca4ca None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:55.870151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031a41-ddbe-46d5-b253-a1402c5ca4ca None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:55.870257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031a41-ddbe-46d5-b253-a1402c5ca4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:55.934808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-88031a41-ddbe-46d5-b253-a1402c5ca4ca None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:55.935982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-88031a41-ddbe-46d5-b253-a1402c5ca4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:55.980736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:56.029580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 746/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:16:56.316250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed914b-cb3f-40db-b31c-6317d9352e83 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:56.318326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e2ed914b-cb3f-40db-b31c-6317d9352e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:56.342354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:56.421247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed914b-cb3f-40db-b31c-6317d9352e83 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:56.494892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09feaec3-5c0c-4fb2-b381-5d701d719e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:56.495418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09feaec3-5c0c-4fb2-b381-5d701d719e66 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:56.495692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09feaec3-5c0c-4fb2-b381-5d701d719e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:56.495960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09feaec3-5c0c-4fb2-b381-5d701d719e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:56.497580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-09feaec3-5c0c-4fb2-b381-5d701d719e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:56.570488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:56.608379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09feaec3-5c0c-4fb2-b381-5d701d719e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:56.609877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09feaec3-5c0c-4fb2-b381-5d701d719e66 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:56.627376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 738/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:16:58.809861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a29f0-dee1-49f5-b141-9b559edb6dc1 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:58.810411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a29f0-dee1-49f5-b141-9b559edb6dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:58.810411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a29f0-dee1-49f5-b141-9b559edb6dc1 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:58.811683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 747/1485] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:16:58.817873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73de005-d4eb-4556-ad73-4f6d9d7674ed None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:58.818324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73de005-d4eb-4556-ad73-4f6d9d7674ed None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:58.818426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73de005-d4eb-4556-ad73-4f6d9d7674ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:58.883735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e73de005-d4eb-4556-ad73-4f6d9d7674ed None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:16:58.884930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e73de005-d4eb-4556-ad73-4f6d9d7674ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:58.979436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 739/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:16:59.268513 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a29f0-dee1-49f5-b141-9b559edb6dc1 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:16:59.270088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0b7a29f0-dee1-49f5-b141-9b559edb6dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:59.374737 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a29f0-dee1-49f5-b141-9b559edb6dc1 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:16:59.445753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:16:59.447195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40880a05-4ed3-497d-8a4b-399c0fbeb0ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:16:59.447787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880a05-4ed3-497d-8a4b-399c0fbeb0ea service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:16:59.448231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880a05-4ed3-497d-8a4b-399c0fbeb0ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:16:59.448728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880a05-4ed3-497d-8a4b-399c0fbeb0ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:16:59.450580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-40880a05-4ed3-497d-8a4b-399c0fbeb0ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:16:59.558979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40880a05-4ed3-497d-8a4b-399c0fbeb0ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:16:59.560360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40880a05-4ed3-497d-8a4b-399c0fbeb0ea service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:16:59.579989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 748/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:17:01.823028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5548995-7eb5-4bd7-9dce-11df7e451205 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:17:01.823623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5548995-7eb5-4bd7-9dce-11df7e451205 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:17:01.823623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5548995-7eb5-4bd7-9dce-11df7e451205 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:17:01.824893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 740/1488] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:17:01.830754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85953c1c-4c53-463d-9514-00f0cd840f72 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:17:01.831155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85953c1c-4c53-463d-9514-00f0cd840f72 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:17:01.831303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85953c1c-4c53-463d-9514-00f0cd840f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:17:01.898149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-85953c1c-4c53-463d-9514-00f0cd840f72 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:17:01.899349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-85953c1c-4c53-463d-9514-00f0cd840f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:17:01.951598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:17:01.994242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 749/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:17:02.286970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5548995-7eb5-4bd7-9dce-11df7e451205 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:17:02.288483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c5548995-7eb5-4bd7-9dce-11df7e451205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:17:02.325368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:17:02.392722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5548995-7eb5-4bd7-9dce-11df7e451205 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:17:02.462993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8678b-57b4-44de-a186-8f501e826374 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:17:02.463581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8678b-57b4-44de-a186-8f501e826374 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:17:02.463854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8678b-57b4-44de-a186-8f501e826374 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:17:02.464120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8678b-57b4-44de-a186-8f501e826374 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:17:02.465700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-99c8678b-57b4-44de-a186-8f501e826374 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:17:02.547403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:17:02.576752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8678b-57b4-44de-a186-8f501e826374 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:17:02.578209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8678b-57b4-44de-a186-8f501e826374 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:17:02.596045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 741/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:17:04.946035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c834e-c0c3-402d-92ff-bd01d45dde95 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:17:04.946569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c834e-c0c3-402d-92ff-bd01d45dde95 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:17:04.946569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c834e-c0c3-402d-92ff-bd01d45dde95 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:17:04.947967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 750/1491] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 20:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:17:04.953094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec9338-0805-4c2d-8662-6aa59d82ca5b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:17:04.953539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec9338-0805-4c2d-8662-6aa59d82ca5b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:17:04.953658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec9338-0805-4c2d-8662-6aa59d82ca5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:17:05.019278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-59ec9338-0805-4c2d-8662-6aa59d82ca5b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:17:05.020485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-59ec9338-0805-4c2d-8662-6aa59d82ca5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:17:05.115103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 742/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 20:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:17:05.404115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c834e-c0c3-402d-92ff-bd01d45dde95 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:17:05.405684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f04c834e-c0c3-402d-92ff-bd01d45dde95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:17:05.507591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c834e-c0c3-402d-92ff-bd01d45dde95 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:17:05.584292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473215f-9ded-4aca-972f-ee03b87297ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:17:05.585305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473215f-9ded-4aca-972f-ee03b87297ef service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:17:05.585760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473215f-9ded-4aca-972f-ee03b87297ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:17:05.586189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473215f-9ded-4aca-972f-ee03b87297ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:17:05.588390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4473215f-9ded-4aca-972f-ee03b87297ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:17:05.599697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:17:05.700041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473215f-9ded-4aca-972f-ee03b87297ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:17:05.701485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473215f-9ded-4aca-972f-ee03b87297ef service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:17:05.719736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 751/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:17:15.113087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=82 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:17:15.717531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=50 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:24.138008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd06abe-d7c4-4b7d-a8e4-72e2dbac9941 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:24.138555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd06abe-d7c4-4b7d-a8e4-72e2dbac9941 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:24.138555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd06abe-d7c4-4b7d-a8e4-72e2dbac9941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:24.213766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-fdd06abe-d7c4-4b7d-a8e4-72e2dbac9941 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:18:24.214978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fdd06abe-d7c4-4b7d-a8e4-72e2dbac9941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:24.309798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 743/1494] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:18:24.317992 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4349d4d-3c9a-4e65-adf1-048f5832572b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:24.318480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4349d4d-3c9a-4e65-adf1-048f5832572b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:24.318592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4349d4d-3c9a-4e65-adf1-048f5832572b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:24.393006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-b4349d4d-3c9a-4e65-adf1-048f5832572b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:18:24.394159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b4349d4d-3c9a-4e65-adf1-048f5832572b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:24.489851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 752/1495] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:18:24.494455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd06abe-d7c4-4b7d-a8e4-72e2dbac9941 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:24.495714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fdd06abe-d7c4-4b7d-a8e4-72e2dbac9941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:24.507040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd06abe-d7c4-4b7d-a8e4-72e2dbac9941 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:24.567624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:24.601939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:24.602543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:24.602820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:24.603091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:24.603729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:18:24.604028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:24.605035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING py.warnings [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:18:24.605035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: warnings.warn(msg) Oct 05 20:18:24.605035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: Oct 05 20:18:24.605948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:24.616877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 744/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:18:24.625299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4349d4d-3c9a-4e65-adf1-048f5832572b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:24.634950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4349d4d-3c9a-4e65-adf1-048f5832572b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:24.696419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:24.730627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:24.731232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:24.731507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:24.731773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:24.732399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:24.734143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WARNING py.warnings [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:18:24.734143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: warnings.warn(msg) Oct 05 20:18:24.734143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: Oct 05 20:18:24.735333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:24.790879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 753/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:18:24 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:18:24.795007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:24.796507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:24.830088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b1d8f-88ae-4295-8e03-3834ca41ff5f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:24.922900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:24.923493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:24.923764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:24.924027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:24.924682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:24.925712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING py.warnings [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:18:24.925712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: warnings.warn(msg) Oct 05 20:18:24.925712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: Oct 05 20:18:24.927522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:24.971263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 745/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:18:24 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:18:24.975287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:24.976765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:25.038317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bedc23-4dc7-415a-8afe-3736af22f57f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:25.120603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:25.132496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:25.133099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:25.133373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:25.133672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:25.134310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:25.135301 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WARNING py.warnings [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:18:25.135301 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: warnings.warn(msg) Oct 05 20:18:25.135301 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: Oct 05 20:18:25.136207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:25.148609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 754/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:18:25.152385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:25.153939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:25.165358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36e0dd-d62b-4a91-8b56-d16a12808961 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:25.190154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:25.259768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140fecc-c8aa-4f6c-abb3-dc4f03d68fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:25.260298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140fecc-c8aa-4f6c-abb3-dc4f03d68fda admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:25.260571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140fecc-c8aa-4f6c-abb3-dc4f03d68fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:25.260839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140fecc-c8aa-4f6c-abb3-dc4f03d68fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:25.261468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140fecc-c8aa-4f6c-abb3-dc4f03d68fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:25.262513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING py.warnings [None req-d140fecc-c8aa-4f6c-abb3-dc4f03d68fda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:18:25.262513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: warnings.warn(msg) Oct 05 20:18:25.262513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: Oct 05 20:18:25.263395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140fecc-c8aa-4f6c-abb3-dc4f03d68fda admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:25.275275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 746/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:18:25.279491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:25.280980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:25.314561 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940c894-2c70-416e-a546-8083aa274f29 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:25.406811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bd7da-c2c6-4f0a-a5f7-aa0dd921b1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:25.407457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bd7da-c2c6-4f0a-a5f7-aa0dd921b1ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:25.407730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bd7da-c2c6-4f0a-a5f7-aa0dd921b1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:25.407999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bd7da-c2c6-4f0a-a5f7-aa0dd921b1ad admin admin] PATH_INFO: `/v3/projects/5ba6c7c167334b14a84e8ccd12022061/users/c8b333cd139a487cbf0b17b41d0b0057/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:25.429773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bd7da-c2c6-4f0a-a5f7-aa0dd921b1ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba6c7c167334b14a84e8ccd12022061, user_id=c8b333cd139a487cbf0b17b41d0b0057, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:25.430784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WARNING py.warnings [None req-ab5bd7da-c2c6-4f0a-a5f7-aa0dd921b1ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:18:25.430784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: warnings.warn(msg) Oct 05 20:18:25.430784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: Oct 05 20:18:25.432730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bd7da-c2c6-4f0a-a5f7-aa0dd921b1ad admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:25.457941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 755/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:18:25 2022] PUT /identity/v3/projects/5ba6c7c167334b14a84e8ccd12022061/users/c8b333cd139a487cbf0b17b41d0b0057/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:18:25.469955 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82341333-f451-4ef2-9ff2-4915333da01c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:25.470232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82341333-f451-4ef2-9ff2-4915333da01c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:25.470499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82341333-f451-4ef2-9ff2-4915333da01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:25.527692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-82341333-f451-4ef2-9ff2-4915333da01c None None] MFA Rules not processed for user `c8b333cd139a487cbf0b17b41d0b0057`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:18:25.528842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-82341333-f451-4ef2-9ff2-4915333da01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:25.605200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 747/1502] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:18:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:18:25.618972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c534f-b056-4d3a-9f6f-5fe38aed7891 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:25.619256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c534f-b056-4d3a-9f6f-5fe38aed7891 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:25.619531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c534f-b056-4d3a-9f6f-5fe38aed7891 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:25.620856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 756/1503] 10.222.0.23 () {54 vars in 963 bytes} [Wed Oct 5 20:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:18:25.625479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f18a6-97f3-4284-978c-a6ee24d3c634 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:25.625803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f18a6-97f3-4284-978c-a6ee24d3c634 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:25.626074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f18a6-97f3-4284-978c-a6ee24d3c634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:25.656939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:25.702284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-cd6f18a6-97f3-4284-978c-a6ee24d3c634 None None] MFA Rules not processed for user `a6e8bc66986d48aa9dcbc19258ff6e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:18:25.703479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-cd6f18a6-97f3-4284-978c-a6ee24d3c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:25.777229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 748/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Wed Oct 5 20:18:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:18:25.785102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c534f-b056-4d3a-9f6f-5fe38aed7891 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:25.787124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8a9c534f-b056-4d3a-9f6f-5fe38aed7891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:25.839042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:25.893073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c534f-b056-4d3a-9f6f-5fe38aed7891 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:25.966124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:25.966719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:25.966993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:25.967261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:25.968814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:26.067164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:26.068542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:26.072047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:26.086393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 757/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:18:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:18:26.131288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dcf1e-87b5-46ec-841b-e339c6b4f100 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:26.131570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dcf1e-87b5-46ec-841b-e339c6b4f100 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:26.131840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dcf1e-87b5-46ec-841b-e339c6b4f100 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:26.133114 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 749/1506] 10.222.0.23 () {52 vars in 830 bytes} [Wed Oct 5 20:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:18:26.136604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:26.137858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:26.235445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12814111-68d9-42da-9f44-6669c3b8191e service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:26.327674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:26.328231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:26.328519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:26.328768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] PATH_INFO: `/v3/users/c8b333cd139a487cbf0b17b41d0b0057` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:26.341789 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b333cd139a487cbf0b17b41d0b0057)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:26.342791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WARNING py.warnings [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:18:26.342791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: warnings.warn(msg) Oct 05 20:18:26.342791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: Oct 05 20:18:26.344006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:26.353428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:26.408165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b333cd139a487cbf0b17b41d0b0057'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:26.408788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b333cd139a487cbf0b17b41d0b0057'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:26.428999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b333cd139a487cbf0b17b41d0b0057'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:26.438795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b333cd139a487cbf0b17b41d0b0057'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:26.439387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b333cd139a487cbf0b17b41d0b0057'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:26.468481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 758/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:18:26 2022] DELETE /identity/v3/users/c8b333cd139a487cbf0b17b41d0b0057 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:18:26.472962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1767c-1df4-4e8d-bbdc-744367b6130f None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:26.473438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1767c-1df4-4e8d-bbdc-744367b6130f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:26.473571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1767c-1df4-4e8d-bbdc-744367b6130f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:26.530936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-72b1767c-1df4-4e8d-bbdc-744367b6130f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:18:26.532130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-72b1767c-1df4-4e8d-bbdc-744367b6130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:26.607391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:26.619836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 750/1508] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:18:26.630054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:26.631264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:26.735201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19dd05-c15e-4db8-ac01-ed8c5518f94d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:26.789962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:26.811506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e21fc-dc8a-4c25-8b38-20c5f51b70a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:26.812113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e21fc-dc8a-4c25-8b38-20c5f51b70a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:26.812382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e21fc-dc8a-4c25-8b38-20c5f51b70a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:26.812644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e21fc-dc8a-4c25-8b38-20c5f51b70a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:26.814227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1e7e21fc-dc8a-4c25-8b38-20c5f51b70a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:26.922614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e21fc-dc8a-4c25-8b38-20c5f51b70a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:26.924032 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e21fc-dc8a-4c25-8b38-20c5f51b70a6 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:26.942854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 759/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:18:27.361200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1767c-1df4-4e8d-bbdc-744367b6130f None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:18:27.362607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-72b1767c-1df4-4e8d-bbdc-744367b6130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:18:27.374500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1767c-1df4-4e8d-bbdc-744367b6130f None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:18:27.471496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:18:27.472052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:18:27.472328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:18:27.473277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] PATH_INFO: `/v3/projects/5ba6c7c167334b14a84e8ccd12022061` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:18:27.483742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba6c7c167334b14a84e8ccd12022061)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:18:27.484761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING py.warnings [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:18:27.484761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: warnings.warn(msg) Oct 05 20:18:27.484761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: Oct 05 20:18:27.485986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:18:27.542247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba6c7c167334b14a84e8ccd12022061', 'request_id': 'req-7ba38f83-ba09-4132-8f37-e6b40fe581d1'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:27.551678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba6c7c167334b14a84e8ccd12022061', 'request_id': 'req-7ba38f83-ba09-4132-8f37-e6b40fe581d1'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:27.564660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba6c7c167334b14a84e8ccd12022061', 'request_id': 'req-7ba38f83-ba09-4132-8f37-e6b40fe581d1'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:27.588131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] The token cache is being invalidate because project 5ba6c7c167334b14a84e8ccd12022061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:18:27.588454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:27.589098 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7ba38f83-ba09-4132-8f37-e6b40fe581d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:18:27.590440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 751/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:18:27 2022] DELETE /identity/v3/projects/5ba6c7c167334b14a84e8ccd12022061 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:18:36.940379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=62 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:18:37.585957 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:19:51.722329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ad2b-377a-493f-9716-445797ede46b None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:19:51.722937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ad2b-377a-493f-9716-445797ede46b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:19:51.722937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ad2b-377a-493f-9716-445797ede46b None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:19:51.724215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:19:51.728997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca79821-e915-4e46-afb4-ccf205698091 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:19:51.729438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca79821-e915-4e46-afb4-ccf205698091 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:19:51.729558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca79821-e915-4e46-afb4-ccf205698091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:19:51.808912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-0ca79821-e915-4e46-afb4-ccf205698091 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:19:51.810132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0ca79821-e915-4e46-afb4-ccf205698091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:19:51.913961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:19:52.068670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2ad2b-377a-493f-9716-445797ede46b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:19:52.070792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a1f2ad2b-377a-493f-9716-445797ede46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:19:52.187698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2ad2b-377a-493f-9716-445797ede46b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:19:52.194059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ed556-77bc-44cc-b74a-1a85b1be4209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:19:52.194672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ed556-77bc-44cc-b74a-1a85b1be4209 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:19:52.194953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ed556-77bc-44cc-b74a-1a85b1be4209 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:19:52.195221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ed556-77bc-44cc-b74a-1a85b1be4209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:19:52.203573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ed556-77bc-44cc-b74a-1a85b1be4209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:19:52.205153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ed556-77bc-44cc-b74a-1a85b1be4209 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:19:52.215153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 761/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:19:53.295164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a86ed-4408-4ae4-8b5c-7135fd485374 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:19:53.295728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a86ed-4408-4ae4-8b5c-7135fd485374 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:19:53.295728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a86ed-4408-4ae4-8b5c-7135fd485374 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:19:53.297050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:19:53.303889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea7222-fd81-4caf-8a97-5be2811bf13e None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:19:53.304309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea7222-fd81-4caf-8a97-5be2811bf13e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:19:53.304464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea7222-fd81-4caf-8a97-5be2811bf13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:19:53.324488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-00ea7222-fd81-4caf-8a97-5be2811bf13e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:19:53.325740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-00ea7222-fd81-4caf-8a97-5be2811bf13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:19:53.334097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:19:53.485465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a86ed-4408-4ae4-8b5c-7135fd485374 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:19:53.499462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a86ed-4408-4ae4-8b5c-7135fd485374 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:19:53.505281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef93a5-9144-4c9b-bcc1-f04b9b1a3415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:19:53.505908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef93a5-9144-4c9b-bcc1-f04b9b1a3415 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:19:53.506184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef93a5-9144-4c9b-bcc1-f04b9b1a3415 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:19:53.506453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef93a5-9144-4c9b-bcc1-f04b9b1a3415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:19:53.514286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef93a5-9144-4c9b-bcc1-f04b9b1a3415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:19:53.515863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef93a5-9144-4c9b-bcc1-f04b9b1a3415 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:19:53.525604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 754/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:19:53.618487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f770866-c5e3-41f6-9f8b-516aa7263079 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:19:53.618793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f770866-c5e3-41f6-9f8b-516aa7263079 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:19:53.619061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f770866-c5e3-41f6-9f8b-516aa7263079 None None] PATH_INFO: `/v3` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:19:53.620551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 763/1517] 10.222.0.15 () {54 vars in 925 bytes} [Wed Oct 5 20:19:53 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:19:53.626673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df295ff-bb62-4379-b2ae-c7d15ce13965 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:19:53.626954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df295ff-bb62-4379-b2ae-c7d15ce13965 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:19:53.627226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df295ff-bb62-4379-b2ae-c7d15ce13965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:19:53.689376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-0df295ff-bb62-4379-b2ae-c7d15ce13965 None None] MFA Rules not processed for user `5181c4b31ac240469aff7a0aee35eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:19:53.690598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0df295ff-bb62-4379-b2ae-c7d15ce13965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:19:53.708204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:19:53.763060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 755/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Oct 5 20:19:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:20:02.206414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfa3a6-1c3c-43d7-974d-e23f8941a8ec None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:02.206903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfa3a6-1c3c-43d7-974d-e23f8941a8ec None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:02.207011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfa3a6-1c3c-43d7-974d-e23f8941a8ec None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:02.208242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 764/1519] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 20:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:20:02.213209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b702-4fa5-4cc3-99a3-40591acc4b48 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:02.213663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b702-4fa5-4cc3-99a3-40591acc4b48 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:02.213884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b702-4fa5-4cc3-99a3-40591acc4b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:02.233343 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-38e0b702-4fa5-4cc3-99a3-40591acc4b48 None None] MFA Rules not processed for user `5181c4b31ac240469aff7a0aee35eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:20:02.234568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-38e0b702-4fa5-4cc3-99a3-40591acc4b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:02.243474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 756/1520] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 20:20:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:20:02.250837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfa3a6-1c3c-43d7-974d-e23f8941a8ec None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:20:02.264668 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfa3a6-1c3c-43d7-974d-e23f8941a8ec None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:20:02.269492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3250f6-56c2-4e18-97b4-e4e47ee5b491 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:20:02.270129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3250f6-56c2-4e18-97b4-e4e47ee5b491 service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:02.270403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3250f6-56c2-4e18-97b4-e4e47ee5b491 service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:02.271262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3250f6-56c2-4e18-97b4-e4e47ee5b491 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:02.279313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3250f6-56c2-4e18-97b4-e4e47ee5b491 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:20:02.280881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3250f6-56c2-4e18-97b4-e4e47ee5b491 service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:20:02.290315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 765/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:20:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:20:05.000781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d36f08-0ca8-44ee-935a-28e6d0d9fe56 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:05.001433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d36f08-0ca8-44ee-935a-28e6d0d9fe56 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:05.001433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d36f08-0ca8-44ee-935a-28e6d0d9fe56 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:05.003100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:20:05.009377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e856a-4119-4ca2-a00d-c271ae7178f4 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:05.009836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e856a-4119-4ca2-a00d-c271ae7178f4 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:05.009970 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e856a-4119-4ca2-a00d-c271ae7178f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:05.029158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-868e856a-4119-4ca2-a00d-c271ae7178f4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:20:05.030376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-868e856a-4119-4ca2-a00d-c271ae7178f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:05.038722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:20:05.190197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d36f08-0ca8-44ee-935a-28e6d0d9fe56 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:20:05.204813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d36f08-0ca8-44ee-935a-28e6d0d9fe56 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:20:05.210715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df56a36-2691-4e7d-bbba-dc682996c685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:20:05.211292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df56a36-2691-4e7d-bbba-dc682996c685 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:05.211566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df56a36-2691-4e7d-bbba-dc682996c685 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:05.211842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df56a36-2691-4e7d-bbba-dc682996c685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:05.219884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df56a36-2691-4e7d-bbba-dc682996c685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:20:05.222147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df56a36-2691-4e7d-bbba-dc682996c685 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:20:05.231389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 758/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:20:06.285774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb4bf9-a79b-4b0a-8909-bac68da7638f None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:06.286316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb4bf9-a79b-4b0a-8909-bac68da7638f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:06.286316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb4bf9-a79b-4b0a-8909-bac68da7638f None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:06.287563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 20:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:20:06.293757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154d801-7f90-42ad-b988-3007b28c0490 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:06.294181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154d801-7f90-42ad-b988-3007b28c0490 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:06.294289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154d801-7f90-42ad-b988-3007b28c0490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:06.313434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-0154d801-7f90-42ad-b988-3007b28c0490 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:20:06.315150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0154d801-7f90-42ad-b988-3007b28c0490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:06.323084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 20:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:20:06.474588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb4bf9-a79b-4b0a-8909-bac68da7638f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:20:06.489237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb4bf9-a79b-4b0a-8909-bac68da7638f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:20:06.494695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61edc16-d594-423c-a382-ac5558676c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:20:06.495277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61edc16-d594-423c-a382-ac5558676c0a service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:06.495550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61edc16-d594-423c-a382-ac5558676c0a service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:06.495817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61edc16-d594-423c-a382-ac5558676c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:06.503724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61edc16-d594-423c-a382-ac5558676c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:20:06.505284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61edc16-d594-423c-a382-ac5558676c0a service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:20:06.514486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 768/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:20:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:20:10.481822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89f6c5-7351-40fe-afba-f7084feffa70 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:10.482346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89f6c5-7351-40fe-afba-f7084feffa70 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:10.482346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89f6c5-7351-40fe-afba-f7084feffa70 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:10.483695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:20:10.488874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acf980-95d2-4b6d-b360-7e16389b36be None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:10.489279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acf980-95d2-4b6d-b360-7e16389b36be None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:10.489413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acf980-95d2-4b6d-b360-7e16389b36be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:10.508668 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-d4acf980-95d2-4b6d-b360-7e16389b36be None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:20:10.509896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d4acf980-95d2-4b6d-b360-7e16389b36be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:10.518133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:20:10.803171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89f6c5-7351-40fe-afba-f7084feffa70 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:20:10.804853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1f89f6c5-7351-40fe-afba-f7084feffa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:10.884205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89f6c5-7351-40fe-afba-f7084feffa70 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:20:10.887870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef62b6-6781-4b62-8747-da6f4e245402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:20:10.888476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef62b6-6781-4b62-8747-da6f4e245402 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:10.888779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef62b6-6781-4b62-8747-da6f4e245402 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:10.889047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef62b6-6781-4b62-8747-da6f4e245402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:10.898512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef62b6-6781-4b62-8747-da6f4e245402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:20:10.899937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef62b6-6781-4b62-8747-da6f4e245402 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:20:10.909190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 761/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:20:13.045965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb14d9-b1c9-409e-be72-c8a8194da1a0 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:13.046468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb14d9-b1c9-409e-be72-c8a8194da1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:13.046468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb14d9-b1c9-409e-be72-c8a8194da1a0 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:13.047873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:20:13.053468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdca03e-44c7-4ac9-808e-f431fd243a9d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:13.053950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdca03e-44c7-4ac9-808e-f431fd243a9d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:13.054055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdca03e-44c7-4ac9-808e-f431fd243a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:13.073255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-bbdca03e-44c7-4ac9-808e-f431fd243a9d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:20:13.074465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-bbdca03e-44c7-4ac9-808e-f431fd243a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:13.082196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:20:13.375663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb14d9-b1c9-409e-be72-c8a8194da1a0 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:20:13.387101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb14d9-b1c9-409e-be72-c8a8194da1a0 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:20:13.390592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993cc2c-f0cc-42c4-98b3-8df14bb175ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:20:13.391199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993cc2c-f0cc-42c4-98b3-8df14bb175ff service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:13.391477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993cc2c-f0cc-42c4-98b3-8df14bb175ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:13.391744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993cc2c-f0cc-42c4-98b3-8df14bb175ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:13.400457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993cc2c-f0cc-42c4-98b3-8df14bb175ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:20:13.401906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993cc2c-f0cc-42c4-98b3-8df14bb175ff service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:20:13.411037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 771/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:20:17.050961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d684c-3406-4cc1-8e61-db45dc284cd3 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:17.051510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d684c-3406-4cc1-8e61-db45dc284cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:17.051510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d684c-3406-4cc1-8e61-db45dc284cd3 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:17.052827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:20:17.058069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b761-2d62-492b-9e75-56847507c1aa None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:17.058473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b761-2d62-492b-9e75-56847507c1aa None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:17.058643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b761-2d62-492b-9e75-56847507c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:17.067951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=81 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:20:17.081198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f6e2b761-2d62-492b-9e75-56847507c1aa None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:20:17.082416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f6e2b761-2d62-492b-9e75-56847507c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:17.090101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:20:17.398047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d684c-3406-4cc1-8e61-db45dc284cd3 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:20:17.399702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5e3d684c-3406-4cc1-8e61-db45dc284cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:17.463690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:20:17.477259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d684c-3406-4cc1-8e61-db45dc284cd3 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:20:17.480890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2fac2-4612-4413-892f-3cdd8ff77c17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:20:17.481494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2fac2-4612-4413-892f-3cdd8ff77c17 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:17.481800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2fac2-4612-4413-892f-3cdd8ff77c17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:17.482072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2fac2-4612-4413-892f-3cdd8ff77c17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:17.490691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2fac2-4612-4413-892f-3cdd8ff77c17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:20:17.492137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2fac2-4612-4413-892f-3cdd8ff77c17 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:20:17.501461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 764/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:20:19.645366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a22df-6541-458f-8232-6c2375a88aec None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:19.646037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a22df-6541-458f-8232-6c2375a88aec None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:19.646037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a22df-6541-458f-8232-6c2375a88aec None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:19.647299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 20:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:20:19.653007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a688148-67d2-4a28-a9ed-ec876c13c571 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:19.653459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a688148-67d2-4a28-a9ed-ec876c13c571 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:19.653659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a688148-67d2-4a28-a9ed-ec876c13c571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:19.674022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1a688148-67d2-4a28-a9ed-ec876c13c571 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:20:19.675219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1a688148-67d2-4a28-a9ed-ec876c13c571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:19.683077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 20:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:20:19.980588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a22df-6541-458f-8232-6c2375a88aec None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:20:19.982314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ff1a22df-6541-458f-8232-6c2375a88aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:20:19.995528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a22df-6541-458f-8232-6c2375a88aec None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:20:19.998939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff45225-7c3f-41b4-86b7-bbb469f4aa13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:20:19.999518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff45225-7c3f-41b4-86b7-bbb469f4aa13 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:20:19.999792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff45225-7c3f-41b4-86b7-bbb469f4aa13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:20:20.000059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff45225-7c3f-41b4-86b7-bbb469f4aa13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:20:20.008803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff45225-7c3f-41b4-86b7-bbb469f4aa13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:20:20.010238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff45225-7c3f-41b4-86b7-bbb469f4aa13 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:20:20.019746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 774/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:20:29.669226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=11 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:20:30.015276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=8 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:40.278528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d08fb4-e2ee-4d3c-96c9-df63b82fea21 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.279636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d08fb4-e2ee-4d3c-96c9-df63b82fea21 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.280202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d08fb4-e2ee-4d3c-96c9-df63b82fea21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.310635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d5d08fb4-e2ee-4d3c-96c9-df63b82fea21 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:40.312835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d5d08fb4-e2ee-4d3c-96c9-df63b82fea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:40.321906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 766/1540] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:40.331488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b085cab-48ac-4a40-bdbd-fc64b9481c99 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.333210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b085cab-48ac-4a40-bdbd-fc64b9481c99 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.333210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b085cab-48ac-4a40-bdbd-fc64b9481c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.362649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-0b085cab-48ac-4a40-bdbd-fc64b9481c99 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:40.363897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0b085cab-48ac-4a40-bdbd-fc64b9481c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:40.380066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 775/1541] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:40.385758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d08fb4-e2ee-4d3c-96c9-df63b82fea21 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:40.400617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d08fb4-e2ee-4d3c-96c9-df63b82fea21 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:40.409669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b61926-a121-497f-9aed-4b636e8ec526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:40.410695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61926-a121-497f-9aed-4b636e8ec526 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.411207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61926-a121-497f-9aed-4b636e8ec526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.411707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61926-a121-497f-9aed-4b636e8ec526 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.412605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61926-a121-497f-9aed-4b636e8ec526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:40.413129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61926-a121-497f-9aed-4b636e8ec526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:40.415030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61926-a121-497f-9aed-4b636e8ec526 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:40.432702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 767/1542] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:21:40.441096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b085cab-48ac-4a40-bdbd-fc64b9481c99 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:40.451166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b085cab-48ac-4a40-bdbd-fc64b9481c99 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:40.464278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eeb94c-79d3-4d39-aa9b-72ec0acb9bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:40.465004 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeb94c-79d3-4d39-aa9b-72ec0acb9bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.465281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeb94c-79d3-4d39-aa9b-72ec0acb9bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.465562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeb94c-79d3-4d39-aa9b-72ec0acb9bd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.466202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eeb94c-79d3-4d39-aa9b-72ec0acb9bd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:40.468907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eeb94c-79d3-4d39-aa9b-72ec0acb9bd9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:40.529325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 776/1543] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:40 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:40.534345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b61926-a121-497f-9aed-4b636e8ec526 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:40.544974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b61926-a121-497f-9aed-4b636e8ec526 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:40.626234 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a70539-2cb7-46ea-aa00-32674830e4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:40.627245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a70539-2cb7-46ea-aa00-32674830e4cd admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.627776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a70539-2cb7-46ea-aa00-32674830e4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.628268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a70539-2cb7-46ea-aa00-32674830e4cd admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.629140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a70539-2cb7-46ea-aa00-32674830e4cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:40.631320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a70539-2cb7-46ea-aa00-32674830e4cd admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:40.667702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 768/1544] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:40 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:40.672436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eeb94c-79d3-4d39-aa9b-72ec0acb9bd9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:40.687472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eeb94c-79d3-4d39-aa9b-72ec0acb9bd9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:40.700060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f8c64-397a-4afc-b232-e9edf7f39a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:40.700848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f8c64-397a-4afc-b232-e9edf7f39a15 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.701126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f8c64-397a-4afc-b232-e9edf7f39a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.701391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f8c64-397a-4afc-b232-e9edf7f39a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.702812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f8c64-397a-4afc-b232-e9edf7f39a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:40.704612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f8c64-397a-4afc-b232-e9edf7f39a15 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:40.721001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 777/1545] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:40.725313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a70539-2cb7-46ea-aa00-32674830e4cd admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:40.739839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a70539-2cb7-46ea-aa00-32674830e4cd admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:40.752564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa7cdc-2182-4cb0-878d-0e4bab9eb034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:40.753659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa7cdc-2182-4cb0-878d-0e4bab9eb034 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.754154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa7cdc-2182-4cb0-878d-0e4bab9eb034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.754662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa7cdc-2182-4cb0-878d-0e4bab9eb034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.755542 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa7cdc-2182-4cb0-878d-0e4bab9eb034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:40.757578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa7cdc-2182-4cb0-878d-0e4bab9eb034 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:40.773909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 769/1546] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:40.774239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f8c64-397a-4afc-b232-e9edf7f39a15 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:40.789794 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f8c64-397a-4afc-b232-e9edf7f39a15 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:40.797709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c1798-634b-4631-9acd-98f4024c1334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:40.798400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1798-634b-4631-9acd-98f4024c1334 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.798677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1798-634b-4631-9acd-98f4024c1334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.798947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c1798-634b-4631-9acd-98f4024c1334 admin admin] PATH_INFO: `/v3/projects/fbc6ca1549a6487cbcbc8ef395cbc801/users/04c46460cf0c478fbf21499f6cceaf65/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.822046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c1798-634b-4631-9acd-98f4024c1334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc6ca1549a6487cbcbc8ef395cbc801, user_id=04c46460cf0c478fbf21499f6cceaf65, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:40.825039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c1798-634b-4631-9acd-98f4024c1334 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:40.832638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5a8ea-08d3-40fc-8c92-205bd09b4295 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.833299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5a8ea-08d3-40fc-8c92-205bd09b4295 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.833824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5a8ea-08d3-40fc-8c92-205bd09b4295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.865060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 778/1547] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:40 2022] PUT /identity/v3/projects/fbc6ca1549a6487cbcbc8ef395cbc801/users/04c46460cf0c478fbf21499f6cceaf65/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:21:40.893805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c2c5a8ea-08d3-40fc-8c92-205bd09b4295 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:40.895413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c2c5a8ea-08d3-40fc-8c92-205bd09b4295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:40.899111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66130cc8-e9f3-493a-b4fc-82085177dd15 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:40.899727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66130cc8-e9f3-493a-b4fc-82085177dd15 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:40.900231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66130cc8-e9f3-493a-b4fc-82085177dd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:40.972028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-66130cc8-e9f3-493a-b4fc-82085177dd15 None None] MFA Rules not processed for user `04c46460cf0c478fbf21499f6cceaf65`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:40.976877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-66130cc8-e9f3-493a-b4fc-82085177dd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.011288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 770/1548] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:21:41.021020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0e189-7ece-40a8-93fd-a2c4854b371d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.021619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0e189-7ece-40a8-93fd-a2c4854b371d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.022184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0e189-7ece-40a8-93fd-a2c4854b371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.056232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-73f0e189-7ece-40a8-93fd-a2c4854b371d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.057959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-73f0e189-7ece-40a8-93fd-a2c4854b371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.075327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 771/1549] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:21:41.083395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0e189-7ece-40a8-93fd-a2c4854b371d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.089095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 779/1550] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.099934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0e189-7ece-40a8-93fd-a2c4854b371d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.109388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bc900-3ea8-41c7-976a-bfea1eb70c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.110432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bc900-3ea8-41c7-976a-bfea1eb70c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.110920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bc900-3ea8-41c7-976a-bfea1eb70c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.111407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bc900-3ea8-41c7-976a-bfea1eb70c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.112260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bc900-3ea8-41c7-976a-bfea1eb70c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:41.124566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0fc3a-0fd5-48ac-a799-642003f69785 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.125100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0fc3a-0fd5-48ac-a799-642003f69785 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.129455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0fc3a-0fd5-48ac-a799-642003f69785 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.131370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 780/1551] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 20:21:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:21:41.133998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bc900-3ea8-41c7-976a-bfea1eb70c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.135907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bc900-3ea8-41c7-976a-bfea1eb70c39 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.144410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb5a4e-7edf-40b9-8955-23ada0e6dd30 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.144944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb5a4e-7edf-40b9-8955-23ada0e6dd30 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.145441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb5a4e-7edf-40b9-8955-23ada0e6dd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.181215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 772/1552] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:21:41.190870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11a226-ebba-48ba-9834-94ce913e6036 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.191899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11a226-ebba-48ba-9834-94ce913e6036 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.192397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11a226-ebba-48ba-9834-94ce913e6036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.230583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ff11a226-ebba-48ba-9834-94ce913e6036 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.233222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ff11a226-ebba-48ba-9834-94ce913e6036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.236172 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-fbeb5a4e-7edf-40b9-8955-23ada0e6dd30 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.238460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fbeb5a4e-7edf-40b9-8955-23ada0e6dd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.248840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 773/1553] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.253141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11a226-ebba-48ba-9834-94ce913e6036 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.274107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11a226-ebba-48ba-9834-94ce913e6036 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.284887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f24de-475e-4417-ac03-ed1699320a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.286410 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f24de-475e-4417-ac03-ed1699320a69 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.286912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f24de-475e-4417-ac03-ed1699320a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.287400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f24de-475e-4417-ac03-ed1699320a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.288263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f24de-475e-4417-ac03-ed1699320a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.290335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING py.warnings [None req-706f24de-475e-4417-ac03-ed1699320a69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:21:41.290335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: warnings.warn(msg) Oct 05 20:21:41.290335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: Oct 05 20:21:41.292263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f24de-475e-4417-ac03-ed1699320a69 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.323188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 774/1554] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:41.324490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53201b3-8017-4eae-9567-045bc1fc8151 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.324778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53201b3-8017-4eae-9567-045bc1fc8151 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.325043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53201b3-8017-4eae-9567-045bc1fc8151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.327929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:41.343816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 781/1555] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.345108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2252f0-ad8e-46f7-aa76-dedad4ccefd3 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.345381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a53201b3-8017-4eae-9567-045bc1fc8151 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.345503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2252f0-ad8e-46f7-aa76-dedad4ccefd3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.345763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2252f0-ad8e-46f7-aa76-dedad4ccefd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.346623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a53201b3-8017-4eae-9567-045bc1fc8151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.352604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:41.366152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7a2252f0-ad8e-46f7-aa76-dedad4ccefd3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.367330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7a2252f0-ad8e-46f7-aa76-dedad4ccefd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.408160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 775/1556] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.410800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12053a1f-7b45-4a64-8f97-035d7432c628 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.411283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12053a1f-7b45-4a64-8f97-035d7432c628 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.411555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12053a1f-7b45-4a64-8f97-035d7432c628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.426137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 782/1557] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.427761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435cb9-de14-401e-a5d2-ddac80944c57 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.428049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435cb9-de14-401e-a5d2-ddac80944c57 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.428322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21435cb9-de14-401e-a5d2-ddac80944c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.431506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-12053a1f-7b45-4a64-8f97-035d7432c628 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.432693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-12053a1f-7b45-4a64-8f97-035d7432c628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.440496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 776/1558] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.442075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f86af-fb65-49ec-afa4-5184232eae17 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.442353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f86af-fb65-49ec-afa4-5184232eae17 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.442619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f86af-fb65-49ec-afa4-5184232eae17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.450228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-21435cb9-de14-401e-a5d2-ddac80944c57 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.451419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-21435cb9-de14-401e-a5d2-ddac80944c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.459908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 783/1559] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.462869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3303ef6-fe01-494e-8633-cf6f278ff585 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.463024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e19f86af-fb65-49ec-afa4-5184232eae17 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.463269 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3303ef6-fe01-494e-8633-cf6f278ff585 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.463548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3303ef6-fe01-494e-8633-cf6f278ff585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.464071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e19f86af-fb65-49ec-afa4-5184232eae17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.472364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 777/1560] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.473488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f86af-fb65-49ec-afa4-5184232eae17 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.483532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a3303ef6-fe01-494e-8633-cf6f278ff585 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.484717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a3303ef6-fe01-494e-8633-cf6f278ff585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.490785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f86af-fb65-49ec-afa4-5184232eae17 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.492825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 784/1561] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.493497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3303ef6-fe01-494e-8633-cf6f278ff585 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.495712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255707dd-b64d-497a-a695-ad334cad2ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.496324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255707dd-b64d-497a-a695-ad334cad2ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.496594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255707dd-b64d-497a-a695-ad334cad2ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.496866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255707dd-b64d-497a-a695-ad334cad2ed3 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.497503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255707dd-b64d-497a-a695-ad334cad2ed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.499991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255707dd-b64d-497a-a695-ad334cad2ed3 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.525744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 778/1562] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/users => generated 291 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:41.525744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611ff88-fb59-4465-8f41-96d19eda19be None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.525744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611ff88-fb59-4465-8f41-96d19eda19be None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.525744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611ff88-fb59-4465-8f41-96d19eda19be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.543999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-2611ff88-fb59-4465-8f41-96d19eda19be None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.545189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2611ff88-fb59-4465-8f41-96d19eda19be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.554091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 779/1563] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.555270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd27ed2-6b8d-4608-81a2-64f96584e617 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.555592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd27ed2-6b8d-4608-81a2-64f96584e617 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.555956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd27ed2-6b8d-4608-81a2-64f96584e617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.575694 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3303ef6-fe01-494e-8633-cf6f278ff585 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.576455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5fd27ed2-6b8d-4608-81a2-64f96584e617 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.578225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5fd27ed2-6b8d-4608-81a2-64f96584e617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.581712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084099b-f31e-43a3-8c01-c9b0fa1354e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.582352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084099b-f31e-43a3-8c01-c9b0fa1354e4 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.582633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084099b-f31e-43a3-8c01-c9b0fa1354e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.582907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084099b-f31e-43a3-8c01-c9b0fa1354e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.586078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 780/1564] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.586969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd27ed2-6b8d-4608-81a2-64f96584e617 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.596520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd27ed2-6b8d-4608-81a2-64f96584e617 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.601028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4d34e-1b31-478c-8bba-c36a0a0417d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.601626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4d34e-1b31-478c-8bba-c36a0a0417d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.601898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4d34e-1b31-478c-8bba-c36a0a0417d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.602162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4d34e-1b31-478c-8bba-c36a0a0417d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.602790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4d34e-1b31-478c-8bba-c36a0a0417d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:41.603083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4d34e-1b31-478c-8bba-c36a0a0417d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.604700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4d34e-1b31-478c-8bba-c36a0a0417d1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.616049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 781/1565] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:21:41.617276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba1a7-2592-4200-a0bf-b7327806f96b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.617583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba1a7-2592-4200-a0bf-b7327806f96b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.617852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ba1a7-2592-4200-a0bf-b7327806f96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.636912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a20ba1a7-2592-4200-a0bf-b7327806f96b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.638193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a20ba1a7-2592-4200-a0bf-b7327806f96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.646603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 782/1566] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.647704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc82d9-14c1-4040-a2a9-b43dbef3f9aa None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.647988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc82d9-14c1-4040-a2a9-b43dbef3f9aa None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.648253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc82d9-14c1-4040-a2a9-b43dbef3f9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.651145 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084099b-f31e-43a3-8c01-c9b0fa1354e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.652770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084099b-f31e-43a3-8c01-c9b0fa1354e4 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.661806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 785/1567] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:41.663028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a656e-a4b9-4eb8-93ea-4126262ad753 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.663249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a656e-a4b9-4eb8-93ea-4126262ad753 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.663518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a656e-a4b9-4eb8-93ea-4126262ad753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.666781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-68dc82d9-14c1-4040-a2a9-b43dbef3f9aa None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.667942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-68dc82d9-14c1-4040-a2a9-b43dbef3f9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.679201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 783/1568] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.679201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc82d9-14c1-4040-a2a9-b43dbef3f9aa None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.686826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e41a656e-a4b9-4eb8-93ea-4126262ad753 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:41.686992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc82d9-14c1-4040-a2a9-b43dbef3f9aa None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.688011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e41a656e-a4b9-4eb8-93ea-4126262ad753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:41.692043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e846c-f4b1-4c53-9d0e-9f35f482f509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.692630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e846c-f4b1-4c53-9d0e-9f35f482f509 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.692902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e846c-f4b1-4c53-9d0e-9f35f482f509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.693218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e846c-f4b1-4c53-9d0e-9f35f482f509 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.693972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e846c-f4b1-4c53-9d0e-9f35f482f509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.695718 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e846c-f4b1-4c53-9d0e-9f35f482f509 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.696217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 786/1569] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:41.696830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a656e-a4b9-4eb8-93ea-4126262ad753 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.706992 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a656e-a4b9-4eb8-93ea-4126262ad753 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.710769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 784/1570] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:41.710769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e846c-f4b1-4c53-9d0e-9f35f482f509 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.714632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.715277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.715657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.715860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.716444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:41.717117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.717803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WARNING py.warnings [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:21:41.717803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: warnings.warn(msg) Oct 05 20:21:41.717803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: Oct 05 20:21:41.718671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.720644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e846c-f4b1-4c53-9d0e-9f35f482f509 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.725261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e04b66-8619-404e-ba06-7b9983004d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.725864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e04b66-8619-404e-ba06-7b9983004d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.726139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e04b66-8619-404e-ba06-7b9983004d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.726406 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e04b66-8619-404e-ba06-7b9983004d8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.727029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e04b66-8619-404e-ba06-7b9983004d8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:41.727325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e04b66-8619-404e-ba06-7b9983004d8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.728890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e04b66-8619-404e-ba06-7b9983004d8f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.731698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:41.736002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 787/1571] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:21:41.736808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.746962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26654c54-d586-4062-b9eb-feb6e6e80f7e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.749484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 785/1572] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:21:41.750405 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e04b66-8619-404e-ba06-7b9983004d8f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.752167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bc9d3-31f2-479c-a0de-291f2ccfd1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.752742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bc9d3-31f2-479c-a0de-291f2ccfd1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.753018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bc9d3-31f2-479c-a0de-291f2ccfd1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.753284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bc9d3-31f2-479c-a0de-291f2ccfd1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.753936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bc9d3-31f2-479c-a0de-291f2ccfd1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.755807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bc9d3-31f2-479c-a0de-291f2ccfd1d5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.760538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e04b66-8619-404e-ba06-7b9983004d8f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.765655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77fa0-974d-472d-bc4c-cf11dc79311a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.766233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77fa0-974d-472d-bc4c-cf11dc79311a admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.766507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77fa0-974d-472d-bc4c-cf11dc79311a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.766771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d77fa0-974d-472d-bc4c-cf11dc79311a admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.767395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d77fa0-974d-472d-bc4c-cf11dc79311a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:41.767689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d77fa0-974d-472d-bc4c-cf11dc79311a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.769287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d77fa0-974d-472d-bc4c-cf11dc79311a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.781878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 786/1573] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:21:41.782822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77fa0-974d-472d-bc4c-cf11dc79311a admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.784634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 788/1574] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:41.785640 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bc9d3-31f2-479c-a0de-291f2ccfd1d5 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.793011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d77fa0-974d-472d-bc4c-cf11dc79311a admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.849786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d15e24-8dd9-4327-a5bf-5c60f0d23405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.850386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d15e24-8dd9-4327-a5bf-5c60f0d23405 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.850657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d15e24-8dd9-4327-a5bf-5c60f0d23405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.850922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d15e24-8dd9-4327-a5bf-5c60f0d23405 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.851555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d15e24-8dd9-4327-a5bf-5c60f0d23405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:41.851848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d15e24-8dd9-4327-a5bf-5c60f0d23405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.853461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d15e24-8dd9-4327-a5bf-5c60f0d23405 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.857831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bc9d3-31f2-479c-a0de-291f2ccfd1d5 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.861618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:41.863640 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abd6f1-5255-4862-9f56-03593def1f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.864409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd6f1-5255-4862-9f56-03593def1f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.864787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd6f1-5255-4862-9f56-03593def1f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.865158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd6f1-5255-4862-9f56-03593def1f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.866034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abd6f1-5255-4862-9f56-03593def1f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:41.866510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abd6f1-5255-4862-9f56-03593def1f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.868256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2abd6f1-5255-4862-9f56-03593def1f2b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.878276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 787/1575] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:21:41.879822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d15e24-8dd9-4327-a5bf-5c60f0d23405 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.889623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 789/1576] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:21:41.889623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abd6f1-5255-4862-9f56-03593def1f2b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.891422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d15e24-8dd9-4327-a5bf-5c60f0d23405 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.894200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2abd6f1-5255-4862-9f56-03593def1f2b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.896070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134312d8-3eb1-4f91-b644-479e7a67c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.896670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134312d8-3eb1-4f91-b644-479e7a67c97c admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.896944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134312d8-3eb1-4f91-b644-479e7a67c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.897202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134312d8-3eb1-4f91-b644-479e7a67c97c admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.897876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134312d8-3eb1-4f91-b644-479e7a67c97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.899471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134312d8-3eb1-4f91-b644-479e7a67c97c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.900295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3e08b-d7ea-4054-876e-bd457f7545ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.901162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3e08b-d7ea-4054-876e-bd457f7545ce admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.901686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3e08b-d7ea-4054-876e-bd457f7545ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.902280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3e08b-d7ea-4054-876e-bd457f7545ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.903152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3e08b-d7ea-4054-876e-bd457f7545ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.905325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3e08b-d7ea-4054-876e-bd457f7545ce admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.909552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 788/1577] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:41.910336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134312d8-3eb1-4f91-b644-479e7a67c97c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.920050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134312d8-3eb1-4f91-b644-479e7a67c97c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:41.926246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f69b4c-cdc3-4a6b-93fd-c2eb16be8602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:41.926972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f69b4c-cdc3-4a6b-93fd-c2eb16be8602 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:41.927164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f69b4c-cdc3-4a6b-93fd-c2eb16be8602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:41.927391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f69b4c-cdc3-4a6b-93fd-c2eb16be8602 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:41.928108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f69b4c-cdc3-4a6b-93fd-c2eb16be8602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:41.930273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f69b4c-cdc3-4a6b-93fd-c2eb16be8602 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:41.938577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 790/1578] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:41.939681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3e08b-d7ea-4054-876e-bd457f7545ce admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:41.963880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 789/1579] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:41.963880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f69b4c-cdc3-4a6b-93fd-c2eb16be8602 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.008403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3e08b-d7ea-4054-876e-bd457f7545ce admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.021767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f69b4c-cdc3-4a6b-93fd-c2eb16be8602 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.027390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fc21a-8833-4f77-a21b-465e17356e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.028174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fc21a-8833-4f77-a21b-465e17356e7a admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.028449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fc21a-8833-4f77-a21b-465e17356e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.028718 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fc21a-8833-4f77-a21b-465e17356e7a admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.029335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fc21a-8833-4f77-a21b-465e17356e7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.031360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fc21a-8833-4f77-a21b-465e17356e7a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.053623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 790/1580] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.056859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fc21a-8833-4f77-a21b-465e17356e7a admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.059189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:42.065359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fc21a-8833-4f77-a21b-465e17356e7a admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.067181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a20baa-2b35-45bb-9d88-1d600960c8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.068211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a20baa-2b35-45bb-9d88-1d600960c8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.068685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a20baa-2b35-45bb-9d88-1d600960c8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.068960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a20baa-2b35-45bb-9d88-1d600960c8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.069627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a20baa-2b35-45bb-9d88-1d600960c8d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.070609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700e26-eb98-4cc9-ad22-b91adb4d86f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.071199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700e26-eb98-4cc9-ad22-b91adb4d86f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.071476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700e26-eb98-4cc9-ad22-b91adb4d86f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.071578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a20baa-2b35-45bb-9d88-1d600960c8d3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.071746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700e26-eb98-4cc9-ad22-b91adb4d86f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.072389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700e26-eb98-4cc9-ad22-b91adb4d86f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.074347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700e26-eb98-4cc9-ad22-b91adb4d86f2 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.098595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 791/1581] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:41 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.099466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a20baa-2b35-45bb-9d88-1d600960c8d3 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.104182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 791/1582] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.105134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700e26-eb98-4cc9-ad22-b91adb4d86f2 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.160506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a20baa-2b35-45bb-9d88-1d600960c8d3 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.165025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700e26-eb98-4cc9-ad22-b91adb4d86f2 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.165196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56cac1-0f1f-4ff8-8f12-32528fb2ad7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.165766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56cac1-0f1f-4ff8-8f12-32528fb2ad7c admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.166046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56cac1-0f1f-4ff8-8f12-32528fb2ad7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.166336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56cac1-0f1f-4ff8-8f12-32528fb2ad7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.166943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56cac1-0f1f-4ff8-8f12-32528fb2ad7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.168847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56cac1-0f1f-4ff8-8f12-32528fb2ad7c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.169744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18238c38-7850-45b9-b4a0-78ca2df7c00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.170443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18238c38-7850-45b9-b4a0-78ca2df7c00b admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.171252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18238c38-7850-45b9-b4a0-78ca2df7c00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.171524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18238c38-7850-45b9-b4a0-78ca2df7c00b admin admin] PATH_INFO: `/v3/projects/6e970d9fb1b84d6299f48f1fb42d2b1d/users/3c607b476313496b80aa0d92330597f4/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.187812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18238c38-7850-45b9-b4a0-78ca2df7c00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e970d9fb1b84d6299f48f1fb42d2b1d, user_id=3c607b476313496b80aa0d92330597f4, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.189043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING py.warnings [None req-18238c38-7850-45b9-b4a0-78ca2df7c00b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:21:42.189043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: warnings.warn(msg) Oct 05 20:21:42.189043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: Oct 05 20:21:42.191309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18238c38-7850-45b9-b4a0-78ca2df7c00b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.199007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 792/1583] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.199832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56cac1-0f1f-4ff8-8f12-32528fb2ad7c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.207204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 792/1584] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:42 2022] PUT /identity/v3/projects/6e970d9fb1b84d6299f48f1fb42d2b1d/users/3c607b476313496b80aa0d92330597f4/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:42.208489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994fb70-e2d4-4213-91c5-d6fc2e7424d6 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.208766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994fb70-e2d4-4213-91c5-d6fc2e7424d6 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.209032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994fb70-e2d4-4213-91c5-d6fc2e7424d6 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.210518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 793/1585] 10.222.0.23 () {54 vars in 912 bytes} [Wed Oct 5 20:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:21:42.211092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994fb70-e2d4-4213-91c5-d6fc2e7424d6 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.230207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:42.274130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56cac1-0f1f-4ff8-8f12-32528fb2ad7c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.278651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994fb70-e2d4-4213-91c5-d6fc2e7424d6 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.278820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d494caf-533b-42d7-8c2b-054e641b1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.279368 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d494caf-533b-42d7-8c2b-054e641b1fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.279648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d494caf-533b-42d7-8c2b-054e641b1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.279937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d494caf-533b-42d7-8c2b-054e641b1fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.280554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d494caf-533b-42d7-8c2b-054e641b1fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.281572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WARNING py.warnings [None req-0d494caf-533b-42d7-8c2b-054e641b1fe2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:21:42.281572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: warnings.warn(msg) Oct 05 20:21:42.281572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: Oct 05 20:21:42.282770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d494caf-533b-42d7-8c2b-054e641b1fe2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.283484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8a91b-7241-4e23-a66b-6a2d1d5731ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.284083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8a91b-7241-4e23-a66b-6a2d1d5731ed admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.284357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8a91b-7241-4e23-a66b-6a2d1d5731ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.284623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8a91b-7241-4e23-a66b-6a2d1d5731ed admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.285237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8a91b-7241-4e23-a66b-6a2d1d5731ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.287157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8a91b-7241-4e23-a66b-6a2d1d5731ed admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.304424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 793/1586] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.305473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d494caf-533b-42d7-8c2b-054e641b1fe2 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.309829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 794/1587] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.310935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8a91b-7241-4e23-a66b-6a2d1d5731ed admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.318096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d494caf-533b-42d7-8c2b-054e641b1fe2 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.321420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8a91b-7241-4e23-a66b-6a2d1d5731ed admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.323048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d9626-3ab5-4efe-81ab-ad040ae529ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.323714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d9626-3ab5-4efe-81ab-ad040ae529ad admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.323988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d9626-3ab5-4efe-81ab-ad040ae529ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.324253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d9626-3ab5-4efe-81ab-ad040ae529ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.324912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d9626-3ab5-4efe-81ab-ad040ae529ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.326091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d4efb-4c04-46f5-91bf-266651489a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.326532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d9626-3ab5-4efe-81ab-ad040ae529ad admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.326656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d4efb-4c04-46f5-91bf-266651489a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.327556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d4efb-4c04-46f5-91bf-266651489a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.327829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d4efb-4c04-46f5-91bf-266651489a10 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.328451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d4efb-4c04-46f5-91bf-266651489a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.330404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d4efb-4c04-46f5-91bf-266651489a10 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.339843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 794/1588] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.339843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d9626-3ab5-4efe-81ab-ad040ae529ad admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.349020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d9626-3ab5-4efe-81ab-ad040ae529ad admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.352409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 795/1589] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.353281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d4efb-4c04-46f5-91bf-266651489a10 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.355122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c73a4-a6e5-42ad-8ab9-685c24e73192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.355777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c73a4-a6e5-42ad-8ab9-685c24e73192 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.356094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c73a4-a6e5-42ad-8ab9-685c24e73192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.356389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c73a4-a6e5-42ad-8ab9-685c24e73192 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.357040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c73a4-a6e5-42ad-8ab9-685c24e73192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.359296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c73a4-a6e5-42ad-8ab9-685c24e73192 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.363391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d4efb-4c04-46f5-91bf-266651489a10 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.368134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3dd5f-9f71-4b19-995e-e97e9fe16647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.372407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3dd5f-9f71-4b19-995e-e97e9fe16647 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.372407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3dd5f-9f71-4b19-995e-e97e9fe16647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.372407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3dd5f-9f71-4b19-995e-e97e9fe16647 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.372407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3dd5f-9f71-4b19-995e-e97e9fe16647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.372407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f3dd5f-9f71-4b19-995e-e97e9fe16647 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.382073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 795/1590] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/users => generated 302 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.383415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc619af-69f4-4db6-9011-4d048c27f0a3 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.383795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc619af-69f4-4db6-9011-4d048c27f0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.384074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc619af-69f4-4db6-9011-4d048c27f0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.397362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 796/1591] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.398735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66191f12-e5f8-4307-9ab6-49d1f23c459a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.399021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66191f12-e5f8-4307-9ab6-49d1f23c459a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.399291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66191f12-e5f8-4307-9ab6-49d1f23c459a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.408462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7dc619af-69f4-4db6-9011-4d048c27f0a3 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:42.409698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7dc619af-69f4-4db6-9011-4d048c27f0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:42.426054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-66191f12-e5f8-4307-9ab6-49d1f23c459a None None] MFA Rules not processed for user `3c607b476313496b80aa0d92330597f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:42.427310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-66191f12-e5f8-4307-9ab6-49d1f23c459a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:42.433339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:42.481549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 796/1592] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:42.482391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc619af-69f4-4db6-9011-4d048c27f0a3 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.492228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc619af-69f4-4db6-9011-4d048c27f0a3 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.496753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3911c2-11ad-4ff8-953a-bb92edbe8df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.497381 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3911c2-11ad-4ff8-953a-bb92edbe8df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.497692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3911c2-11ad-4ff8-953a-bb92edbe8df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.497961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3911c2-11ad-4ff8-953a-bb92edbe8df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.498609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3911c2-11ad-4ff8-953a-bb92edbe8df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.500711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3911c2-11ad-4ff8-953a-bb92edbe8df3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.501948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 797/1593] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:42.502858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66191f12-e5f8-4307-9ab6-49d1f23c459a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.513033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66191f12-e5f8-4307-9ab6-49d1f23c459a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.518145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9ae42-768b-4117-b88b-1eb70fb8fdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.518745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9ae42-768b-4117-b88b-1eb70fb8fdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.518936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 797/1594] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.519755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3911c2-11ad-4ff8-953a-bb92edbe8df3 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.521651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9ae42-768b-4117-b88b-1eb70fb8fdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.521973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9ae42-768b-4117-b88b-1eb70fb8fdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.522694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9ae42-768b-4117-b88b-1eb70fb8fdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.524326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9ae42-768b-4117-b88b-1eb70fb8fdb9 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.529286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3911c2-11ad-4ff8-953a-bb92edbe8df3 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.534044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320313c-4b33-45d7-a8af-e5509a0ca411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.534630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320313c-4b33-45d7-a8af-e5509a0ca411 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.534908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320313c-4b33-45d7-a8af-e5509a0ca411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.535164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 798/1595] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.535274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320313c-4b33-45d7-a8af-e5509a0ca411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.535875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320313c-4b33-45d7-a8af-e5509a0ca411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.536232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9ae42-768b-4117-b88b-1eb70fb8fdb9 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.537617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320313c-4b33-45d7-a8af-e5509a0ca411 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.546650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9ae42-768b-4117-b88b-1eb70fb8fdb9 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.550716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 798/1596] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.550716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320313c-4b33-45d7-a8af-e5509a0ca411 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.551401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3c61-4e96-4518-aab0-5cb8640c05bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.551987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3c61-4e96-4518-aab0-5cb8640c05bd admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.552265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3c61-4e96-4518-aab0-5cb8640c05bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.552536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3c61-4e96-4518-aab0-5cb8640c05bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.553162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc3c61-4e96-4518-aab0-5cb8640c05bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.554773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc3c61-4e96-4518-aab0-5cb8640c05bd admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.558314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320313c-4b33-45d7-a8af-e5509a0ca411 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.563566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbded3b-d641-4de5-8bd5-c7bcb139db7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.564192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbded3b-d641-4de5-8bd5-c7bcb139db7d admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.564466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbded3b-d641-4de5-8bd5-c7bcb139db7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.564733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbded3b-d641-4de5-8bd5-c7bcb139db7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.565019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 799/1597] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.565352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbded3b-d641-4de5-8bd5-c7bcb139db7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.565825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3c61-4e96-4518-aab0-5cb8640c05bd admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.567299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbded3b-d641-4de5-8bd5-c7bcb139db7d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.575366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3c61-4e96-4518-aab0-5cb8640c05bd admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.578585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 799/1598] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.580909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2bea3-b3e6-48ef-afbb-8590c450b679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.581694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2bea3-b3e6-48ef-afbb-8590c450b679 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.581978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2bea3-b3e6-48ef-afbb-8590c450b679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.582249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2bea3-b3e6-48ef-afbb-8590c450b679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.582365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbded3b-d641-4de5-8bd5-c7bcb139db7d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.582877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2bea3-b3e6-48ef-afbb-8590c450b679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.584451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2bea3-b3e6-48ef-afbb-8590c450b679 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.595306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 800/1599] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.596231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2bea3-b3e6-48ef-afbb-8590c450b679 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.605994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2bea3-b3e6-48ef-afbb-8590c450b679 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.611574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef5bc6-49d8-4734-afef-c2a7c25df96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.612179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef5bc6-49d8-4734-afef-c2a7c25df96f admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.612455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef5bc6-49d8-4734-afef-c2a7c25df96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.612723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef5bc6-49d8-4734-afef-c2a7c25df96f admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.613335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef5bc6-49d8-4734-afef-c2a7c25df96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.615255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef5bc6-49d8-4734-afef-c2a7c25df96f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.638252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbded3b-d641-4de5-8bd5-c7bcb139db7d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.642050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c6805-9664-40d4-8f4f-ccc2e121e937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.642217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 801/1600] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.643087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef5bc6-49d8-4734-afef-c2a7c25df96f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.645612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c6805-9664-40d4-8f4f-ccc2e121e937 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.645914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c6805-9664-40d4-8f4f-ccc2e121e937 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.646170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c6805-9664-40d4-8f4f-ccc2e121e937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.648382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:42.657014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c6805-9664-40d4-8f4f-ccc2e121e937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.658486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c6805-9664-40d4-8f4f-ccc2e121e937 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.669428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 800/1601] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:42.670271 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c6805-9664-40d4-8f4f-ccc2e121e937 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.680488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c6805-9664-40d4-8f4f-ccc2e121e937 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.711012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef5bc6-49d8-4734-afef-c2a7c25df96f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.715810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f6bcd-c316-4e46-85b2-8e2dba947b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.716426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f6bcd-c316-4e46-85b2-8e2dba947b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.716707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f6bcd-c316-4e46-85b2-8e2dba947b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.716980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f6bcd-c316-4e46-85b2-8e2dba947b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.717686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f6bcd-c316-4e46-85b2-8e2dba947b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.719280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f6bcd-c316-4e46-85b2-8e2dba947b76 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.733602 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 802/1602] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.734455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f6bcd-c316-4e46-85b2-8e2dba947b76 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.741891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694f529-0759-474f-b5b3-c986d21cdd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.742577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694f529-0759-474f-b5b3-c986d21cdd57 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.742924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694f529-0759-474f-b5b3-c986d21cdd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.743259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694f529-0759-474f-b5b3-c986d21cdd57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.744026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694f529-0759-474f-b5b3-c986d21cdd57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.745326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f6bcd-c316-4e46-85b2-8e2dba947b76 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.745770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694f529-0759-474f-b5b3-c986d21cdd57 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.752666 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dd141-040f-4e09-87b1-349b7868e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.753593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dd141-040f-4e09-87b1-349b7868e74d admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.753991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dd141-040f-4e09-87b1-349b7868e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.754279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dd141-040f-4e09-87b1-349b7868e74d admin admin] PATH_INFO: `/v3/projects/e43dabe5f2b34bf29e56413ca0e21886/users/f87d19eadffc4ab6ad22d95f865bce27/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.757151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 801/1603] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.757856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694f529-0759-474f-b5b3-c986d21cdd57 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.767819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694f529-0759-474f-b5b3-c986d21cdd57 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.771555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0dd141-040f-4e09-87b1-349b7868e74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43dabe5f2b34bf29e56413ca0e21886, user_id=f87d19eadffc4ab6ad22d95f865bce27, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.773005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c163fc3-bf56-4361-b65b-7bd108771517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.773870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c163fc3-bf56-4361-b65b-7bd108771517 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.774262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c163fc3-bf56-4361-b65b-7bd108771517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.774609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c163fc3-bf56-4361-b65b-7bd108771517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.774750 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0dd141-040f-4e09-87b1-349b7868e74d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.775382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c163fc3-bf56-4361-b65b-7bd108771517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.777225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c163fc3-bf56-4361-b65b-7bd108771517 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.789101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 803/1604] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:42 2022] PUT /identity/v3/projects/e43dabe5f2b34bf29e56413ca0e21886/users/f87d19eadffc4ab6ad22d95f865bce27/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:42.790029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dd141-040f-4e09-87b1-349b7868e74d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.790584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 802/1605] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.791439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c163fc3-bf56-4361-b65b-7bd108771517 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.803087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c163fc3-bf56-4361-b65b-7bd108771517 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.815760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:42.862002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dd141-040f-4e09-87b1-349b7868e74d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.869940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab967e7-20bc-4f8e-870c-f91e6b3d7389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.870555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab967e7-20bc-4f8e-870c-f91e6b3d7389 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.870690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23ac71-4068-4ad0-95d2-2194bd6e143b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.870889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab967e7-20bc-4f8e-870c-f91e6b3d7389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.871091 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab967e7-20bc-4f8e-870c-f91e6b3d7389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.871194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23ac71-4068-4ad0-95d2-2194bd6e143b admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.871480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23ac71-4068-4ad0-95d2-2194bd6e143b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.871733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab967e7-20bc-4f8e-870c-f91e6b3d7389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.871853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23ac71-4068-4ad0-95d2-2194bd6e143b admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.872380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23ac71-4068-4ad0-95d2-2194bd6e143b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.873727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab967e7-20bc-4f8e-870c-f91e6b3d7389 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.875245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23ac71-4068-4ad0-95d2-2194bd6e143b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.886439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 803/1606] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.889895 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 804/1607] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:42.890995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab967e7-20bc-4f8e-870c-f91e6b3d7389 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.891485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23ac71-4068-4ad0-95d2-2194bd6e143b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.900966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab967e7-20bc-4f8e-870c-f91e6b3d7389 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.901564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23ac71-4068-4ad0-95d2-2194bd6e143b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.906091 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67233db8-74a8-44c6-92ab-9af7fe442638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.906679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67233db8-74a8-44c6-92ab-9af7fe442638 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.906955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67233db8-74a8-44c6-92ab-9af7fe442638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.907082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe76c7a-e74b-45c5-9d4c-eaaaa8c6a1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:42.907229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67233db8-74a8-44c6-92ab-9af7fe442638 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.907854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67233db8-74a8-44c6-92ab-9af7fe442638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.907981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe76c7a-e74b-45c5-9d4c-eaaaa8c6a1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:42.908936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe76c7a-e74b-45c5-9d4c-eaaaa8c6a1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:42.908936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe76c7a-e74b-45c5-9d4c-eaaaa8c6a1a6 admin admin] PATH_INFO: `/v3/projects/38145c704ac548abb6747e073892ff72/users/a553cd4c2e154ee1979812ae575e6fb2/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:42.909785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67233db8-74a8-44c6-92ab-9af7fe442638 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.924502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe76c7a-e74b-45c5-9d4c-eaaaa8c6a1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38145c704ac548abb6747e073892ff72, user_id=a553cd4c2e154ee1979812ae575e6fb2, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:42.927845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe76c7a-e74b-45c5-9d4c-eaaaa8c6a1a6 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:42.933008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 804/1608] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:42 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:42.933008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67233db8-74a8-44c6-92ab-9af7fe442638 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:42.944441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67233db8-74a8-44c6-92ab-9af7fe442638 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:42.945951 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 805/1609] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:42 2022] PUT /identity/v3/projects/38145c704ac548abb6747e073892ff72/users/a553cd4c2e154ee1979812ae575e6fb2/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:42.946894 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe76c7a-e74b-45c5-9d4c-eaaaa8c6a1a6 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.004392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:43.020109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43921f-c82b-4644-ab7a-623b6a32169c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.020771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43921f-c82b-4644-ab7a-623b6a32169c admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.021048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43921f-c82b-4644-ab7a-623b6a32169c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.021315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43921f-c82b-4644-ab7a-623b6a32169c admin admin] PATH_INFO: `/v3/projects/07f868da50254a68bc692bca433d2699/users/1301a78c1a684c94acb9f26b56f39f21/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.022181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe76c7a-e74b-45c5-9d4c-eaaaa8c6a1a6 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.028127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4826aaa-23ed-496f-9236-3b833eff1c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.028861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4826aaa-23ed-496f-9236-3b833eff1c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.029235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4826aaa-23ed-496f-9236-3b833eff1c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.029626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4826aaa-23ed-496f-9236-3b833eff1c76 admin admin] PATH_INFO: `/v3/projects/45fc2abd2bae4264b1081c1673e3095e/users/c28866ff5b2a4dd4ba05270b369f064e/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.036507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43921f-c82b-4644-ab7a-623b6a32169c admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f868da50254a68bc692bca433d2699, user_id=1301a78c1a684c94acb9f26b56f39f21, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.039257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43921f-c82b-4644-ab7a-623b6a32169c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.048848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4826aaa-23ed-496f-9236-3b833eff1c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fc2abd2bae4264b1081c1673e3095e, user_id=c28866ff5b2a4dd4ba05270b369f064e, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.051605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4826aaa-23ed-496f-9236-3b833eff1c76 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.054689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 805/1610] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:42 2022] PUT /identity/v3/projects/07f868da50254a68bc692bca433d2699/users/1301a78c1a684c94acb9f26b56f39f21/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:43.056123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25903fd4-7a24-4b12-86e3-819c4b59b534 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.056417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25903fd4-7a24-4b12-86e3-819c4b59b534 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.056687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25903fd4-7a24-4b12-86e3-819c4b59b534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.067989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 806/1611] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:42 2022] PUT /identity/v3/projects/45fc2abd2bae4264b1081c1673e3095e/users/c28866ff5b2a4dd4ba05270b369f064e/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:43.070324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4826aaa-23ed-496f-9236-3b833eff1c76 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.087246 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-25903fd4-7a24-4b12-86e3-819c4b59b534 None None] MFA Rules not processed for user `f87d19eadffc4ab6ad22d95f865bce27`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:43.088464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-25903fd4-7a24-4b12-86e3-819c4b59b534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:43.148528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4826aaa-23ed-496f-9236-3b833eff1c76 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.153478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c5868-aa26-4d2a-a093-3fc5a0af46d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.154530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c5868-aa26-4d2a-a093-3fc5a0af46d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.155042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c5868-aa26-4d2a-a093-3fc5a0af46d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.155608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c5868-aa26-4d2a-a093-3fc5a0af46d3 admin admin] PATH_INFO: `/v3/projects/140a5d8a1d264d8bad17d352591fa6a2/users/ff9b14ab630d4a928614c60e6aea667c/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.157273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 806/1612] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:43.160971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25903fd4-7a24-4b12-86e3-819c4b59b534 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.176909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25903fd4-7a24-4b12-86e3-819c4b59b534 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.180350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c5868-aa26-4d2a-a093-3fc5a0af46d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=140a5d8a1d264d8bad17d352591fa6a2, user_id=ff9b14ab630d4a928614c60e6aea667c, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.182017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd128be-89b6-41df-b51f-674a34ab0a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.182693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd128be-89b6-41df-b51f-674a34ab0a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.183159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd128be-89b6-41df-b51f-674a34ab0a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.183442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd128be-89b6-41df-b51f-674a34ab0a0c admin admin] PATH_INFO: `/v3/projects/46c4e531a28a4944a2beab1d8751a07b/users/cbaa416043eb4b4f9b628625f0cddd6b/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.183555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c5868-aa26-4d2a-a093-3fc5a0af46d3 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.185977 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:43.204790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 807/1613] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:43 2022] PUT /identity/v3/projects/140a5d8a1d264d8bad17d352591fa6a2/users/ff9b14ab630d4a928614c60e6aea667c/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:43.205897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c5868-aa26-4d2a-a093-3fc5a0af46d3 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.210780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd128be-89b6-41df-b51f-674a34ab0a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c4e531a28a4944a2beab1d8751a07b, user_id=cbaa416043eb4b4f9b628625f0cddd6b, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.213904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd128be-89b6-41df-b51f-674a34ab0a0c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.231820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 807/1614] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:43 2022] PUT /identity/v3/projects/46c4e531a28a4944a2beab1d8751a07b/users/cbaa416043eb4b4f9b628625f0cddd6b/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:43.233384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61fc36-fcfa-4a22-b211-11a982a1bc05 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.233826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61fc36-fcfa-4a22-b211-11a982a1bc05 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.234107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61fc36-fcfa-4a22-b211-11a982a1bc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.257761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-0e61fc36-fcfa-4a22-b211-11a982a1bc05 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:43.259083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0e61fc36-fcfa-4a22-b211-11a982a1bc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:43.296183 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c5868-aa26-4d2a-a093-3fc5a0af46d3 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.341002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 808/1615] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:43.342674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaa273-45a2-454b-bf0f-5d4e01f25d8c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.342959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaa273-45a2-454b-bf0f-5d4e01f25d8c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.343230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaa273-45a2-454b-bf0f-5d4e01f25d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.362499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6d1af-5cbe-4085-ae98-e2d30775d842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.363432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6d1af-5cbe-4085-ae98-e2d30775d842 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.363803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6d1af-5cbe-4085-ae98-e2d30775d842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.364173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6d1af-5cbe-4085-ae98-e2d30775d842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.365102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6d1af-5cbe-4085-ae98-e2d30775d842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.367023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6d1af-5cbe-4085-ae98-e2d30775d842 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.369379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:43.374780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-dedaa273-45a2-454b-bf0f-5d4e01f25d8c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:43.375991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-dedaa273-45a2-454b-bf0f-5d4e01f25d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:43.379764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 808/1616] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:43.382933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbf066-6f72-49a8-a3b5-a277e35fe2f5 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.383523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbf066-6f72-49a8-a3b5-a277e35fe2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.383972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbf066-6f72-49a8-a3b5-a277e35fe2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.388505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 809/1617] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:43.391683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaa273-45a2-454b-bf0f-5d4e01f25d8c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.405969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaa273-45a2-454b-bf0f-5d4e01f25d8c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.408792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-23cbf066-6f72-49a8-a3b5-a277e35fe2f5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:43.410402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-23cbf066-6f72-49a8-a3b5-a277e35fe2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:43.417828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ad25-07e8-40dc-bda8-0f98d9d83e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.418579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ad25-07e8-40dc-bda8-0f98d9d83e25 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.422600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ad25-07e8-40dc-bda8-0f98d9d83e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.422732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 809/1618] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:43.422843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ad25-07e8-40dc-bda8-0f98d9d83e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.422843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103ad25-07e8-40dc-bda8-0f98d9d83e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.423032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cd28e-6ad1-47f2-ae3d-ae8cfe9d2bc0 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.423032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cd28e-6ad1-47f2-ae3d-ae8cfe9d2bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.423032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cd28e-6ad1-47f2-ae3d-ae8cfe9d2bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.423379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103ad25-07e8-40dc-bda8-0f98d9d83e25 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.448451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a41cd28e-6ad1-47f2-ae3d-ae8cfe9d2bc0 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:43.449977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a41cd28e-6ad1-47f2-ae3d-ae8cfe9d2bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:43.458761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 810/1619] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:43.459707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 810/1620] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:43.463896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0f9d6-8719-4d4c-824f-259ff6c9b82b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.464179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0f9d6-8719-4d4c-824f-259ff6c9b82b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.464544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0f9d6-8719-4d4c-824f-259ff6c9b82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.466300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ad25-07e8-40dc-bda8-0f98d9d83e25 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.493399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-3fb0f9d6-8719-4d4c-824f-259ff6c9b82b None None] MFA Rules not processed for user `cbaa416043eb4b4f9b628625f0cddd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:43.494646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3fb0f9d6-8719-4d4c-824f-259ff6c9b82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:43.541673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ad25-07e8-40dc-bda8-0f98d9d83e25 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.546186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820b075-a89b-4c0e-a069-6a42df7a5b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.546810 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820b075-a89b-4c0e-a069-6a42df7a5b12 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.547300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820b075-a89b-4c0e-a069-6a42df7a5b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.547582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820b075-a89b-4c0e-a069-6a42df7a5b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.577240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 811/1621] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:43.578601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0f9d6-8719-4d4c-824f-259ff6c9b82b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.616457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820b075-a89b-4c0e-a069-6a42df7a5b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.617935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820b075-a89b-4c0e-a069-6a42df7a5b12 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.628025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 811/1622] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:43.628831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820b075-a89b-4c0e-a069-6a42df7a5b12 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.633658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:43.637750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820b075-a89b-4c0e-a069-6a42df7a5b12 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.645540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f6955-976e-46a7-a055-ac921ae4bd88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.646310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f6955-976e-46a7-a055-ac921ae4bd88 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.646456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0f9d6-8719-4d4c-824f-259ff6c9b82b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.646558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f6955-976e-46a7-a055-ac921ae4bd88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.646666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f6955-976e-46a7-a055-ac921ae4bd88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.651709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f511b0-1944-4091-a0aa-3e70a9364c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.652449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f511b0-1944-4091-a0aa-3e70a9364c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.652715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f511b0-1944-4091-a0aa-3e70a9364c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.653015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f511b0-1944-4091-a0aa-3e70a9364c26 admin admin] PATH_INFO: `/v3/projects/316bdaecc03c4a7d9e9dc91dd32ca85a/users/6fefeaae734b488ba769eec37d398a5d/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.658421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f6955-976e-46a7-a055-ac921ae4bd88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.660030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f6955-976e-46a7-a055-ac921ae4bd88 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.669971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f511b0-1944-4091-a0aa-3e70a9364c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=316bdaecc03c4a7d9e9dc91dd32ca85a, user_id=6fefeaae734b488ba769eec37d398a5d, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.671418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 812/1623] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:43.672271 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f6955-976e-46a7-a055-ac921ae4bd88 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.672818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f511b0-1944-4091-a0aa-3e70a9364c26 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.680567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f6955-976e-46a7-a055-ac921ae4bd88 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.688897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 812/1624] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:43 2022] PUT /identity/v3/projects/316bdaecc03c4a7d9e9dc91dd32ca85a/users/6fefeaae734b488ba769eec37d398a5d/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:43.689807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f511b0-1944-4091-a0aa-3e70a9364c26 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.736941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038bd21-7dd1-495c-b3fc-ab88ae192da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.737585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038bd21-7dd1-495c-b3fc-ab88ae192da9 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.737865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038bd21-7dd1-495c-b3fc-ab88ae192da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.738157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038bd21-7dd1-495c-b3fc-ab88ae192da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.767612 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:43.771106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f511b0-1944-4091-a0aa-3e70a9364c26 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.777734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c257698-b55d-4ba0-8656-e8aa7e849261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.778366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c257698-b55d-4ba0-8656-e8aa7e849261 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.778736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c257698-b55d-4ba0-8656-e8aa7e849261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.779041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c257698-b55d-4ba0-8656-e8aa7e849261 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.779823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c257698-b55d-4ba0-8656-e8aa7e849261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.784517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c257698-b55d-4ba0-8656-e8aa7e849261 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.811366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 813/1625] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:43.812460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c257698-b55d-4ba0-8656-e8aa7e849261 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.816321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6c257698-b55d-4ba0-8656-e8aa7e849261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:43.818387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038bd21-7dd1-495c-b3fc-ab88ae192da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.819925 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038bd21-7dd1-495c-b3fc-ab88ae192da9 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.828755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c257698-b55d-4ba0-8656-e8aa7e849261 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.830115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 813/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:43.831576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618e917-6326-4f85-bf26-5ad0914336fa None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.831951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618e917-6326-4f85-bf26-5ad0914336fa None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.832234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618e917-6326-4f85-bf26-5ad0914336fa None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.832924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e01e-e48b-45b3-8e2d-52a4f8880658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.833762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e01e-e48b-45b3-8e2d-52a4f8880658 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.834187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e01e-e48b-45b3-8e2d-52a4f8880658 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.834515 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e01e-e48b-45b3-8e2d-52a4f8880658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.835218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 814/1627] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 20:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:21:43.836061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618e917-6326-4f85-bf26-5ad0914336fa None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.848133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9e01e-e48b-45b3-8e2d-52a4f8880658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.850244 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9e01e-e48b-45b3-8e2d-52a4f8880658 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.854413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618e917-6326-4f85-bf26-5ad0914336fa None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.862083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9231-eed3-4f61-aa26-3c39fe0e2b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.866352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9231-eed3-4f61-aa26-3c39fe0e2b8a admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.866352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9231-eed3-4f61-aa26-3c39fe0e2b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.866352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9231-eed3-4f61-aa26-3c39fe0e2b8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.866352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b9231-eed3-4f61-aa26-3c39fe0e2b8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.866352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b9231-eed3-4f61-aa26-3c39fe0e2b8a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.870231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 814/1628] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:43.871542 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e01e-e48b-45b3-8e2d-52a4f8880658 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.883607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e01e-e48b-45b3-8e2d-52a4f8880658 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.889429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaef0dd-2d5e-48e5-96fd-c9fbe7e025cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:43.890055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaef0dd-2d5e-48e5-96fd-c9fbe7e025cf admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.890452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaef0dd-2d5e-48e5-96fd-c9fbe7e025cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.891116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaef0dd-2d5e-48e5-96fd-c9fbe7e025cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.891518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaef0dd-2d5e-48e5-96fd-c9fbe7e025cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:43.893209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaef0dd-2d5e-48e5-96fd-c9fbe7e025cf admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:43.911445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 815/1629] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:43.912745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaef0dd-2d5e-48e5-96fd-c9fbe7e025cf admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:43.920198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 815/1630] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/projects => generated 403 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:43.922481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b42eb-25fb-4037-8b3e-67fc2f86c1a3 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:43.922965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b42eb-25fb-4037-8b3e-67fc2f86c1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:43.923311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b42eb-25fb-4037-8b3e-67fc2f86c1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:43.931493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaef0dd-2d5e-48e5-96fd-c9fbe7e025cf admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:43.950605 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c11b42eb-25fb-4037-8b3e-67fc2f86c1a3 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:43.952843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c11b42eb-25fb-4037-8b3e-67fc2f86c1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:44.016467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1384a2c-768d-4991-b451-56623888bced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.017222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384a2c-768d-4991-b451-56623888bced admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.017626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384a2c-768d-4991-b451-56623888bced admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.017948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384a2c-768d-4991-b451-56623888bced admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.018728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1384a2c-768d-4991-b451-56623888bced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:44.021140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1384a2c-768d-4991-b451-56623888bced admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:44.034898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 816/1631] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:44.036700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1384a2c-768d-4991-b451-56623888bced admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.049246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1384a2c-768d-4991-b451-56623888bced admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:44.055058 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ee46f-27fe-431b-93dc-e23a3fd80cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.055683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ee46f-27fe-431b-93dc-e23a3fd80cde admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.055965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ee46f-27fe-431b-93dc-e23a3fd80cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.056307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ee46f-27fe-431b-93dc-e23a3fd80cde admin admin] PATH_INFO: `/v3/projects/316bdaecc03c4a7d9e9dc91dd32ca85a/users/6fefeaae734b488ba769eec37d398a5d/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.058345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ee46f-27fe-431b-93dc-e23a3fd80cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=316bdaecc03c4a7d9e9dc91dd32ca85a, user_id=6fefeaae734b488ba769eec37d398a5d, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:44.061441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ee46f-27fe-431b-93dc-e23a3fd80cde admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:44.075753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 816/1632] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 20:21:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:44.076617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b42eb-25fb-4037-8b3e-67fc2f86c1a3 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.088511 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b42eb-25fb-4037-8b3e-67fc2f86c1a3 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:44.094084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 817/1633] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:44 2022] PUT /identity/v3/projects/316bdaecc03c4a7d9e9dc91dd32ca85a/users/6fefeaae734b488ba769eec37d398a5d/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:44.096038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ee46f-27fe-431b-93dc-e23a3fd80cde admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.099658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:44.178867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008d851-dc5b-48e1-b1d7-6bbddd998a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.179575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008d851-dc5b-48e1-b1d7-6bbddd998a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.179850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008d851-dc5b-48e1-b1d7-6bbddd998a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.180151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008d851-dc5b-48e1-b1d7-6bbddd998a9c admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.182147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008d851-dc5b-48e1-b1d7-6bbddd998a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:44.185741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008d851-dc5b-48e1-b1d7-6bbddd998a9c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:44.220955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 817/1634] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:44 2022] POST /identity/v3/users => generated 308 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:44.220955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36cac-ba12-4507-b648-a9b0533cb66c None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.220955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36cac-ba12-4507-b648-a9b0533cb66c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.220955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd36cac-ba12-4507-b648-a9b0533cb66c None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.221838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 818/1635] 10.222.0.23 () {54 vars in 912 bytes} [Wed Oct 5 20:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:21:44.223034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c82fc0-3dcb-4ba3-98df-459cc551eb5a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.223397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c82fc0-3dcb-4ba3-98df-459cc551eb5a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.223664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c82fc0-3dcb-4ba3-98df-459cc551eb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.240961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ee46f-27fe-431b-93dc-e23a3fd80cde admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:44.249920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5be28-4760-490d-ad41-974cd438e7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.250997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5be28-4760-490d-ad41-974cd438e7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.251477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5be28-4760-490d-ad41-974cd438e7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.252045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5be28-4760-490d-ad41-974cd438e7e5 admin admin] PATH_INFO: `/v3/projects/aaff62847f4d4c82b76ba8cbf27bb02c/users/fbabef64c6344dcaad0a04b482df8139/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.256963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-87c82fc0-3dcb-4ba3-98df-459cc551eb5a None None] MFA Rules not processed for user `6fefeaae734b488ba769eec37d398a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:44.258404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-87c82fc0-3dcb-4ba3-98df-459cc551eb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:44.274851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5be28-4760-490d-ad41-974cd438e7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaff62847f4d4c82b76ba8cbf27bb02c, user_id=fbabef64c6344dcaad0a04b482df8139, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:44.278512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5be28-4760-490d-ad41-974cd438e7e5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:44.301600 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 818/1636] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:44 2022] PUT /identity/v3/projects/aaff62847f4d4c82b76ba8cbf27bb02c/users/fbabef64c6344dcaad0a04b482df8139/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:44.302555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5be28-4760-490d-ad41-974cd438e7e5 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.368557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:44.410522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 819/1637] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:44.411901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2515d6-bf18-4df5-894c-2bedd47a58d5 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.412192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2515d6-bf18-4df5-894c-2bedd47a58d5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.412751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2515d6-bf18-4df5-894c-2bedd47a58d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.439107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5be28-4760-490d-ad41-974cd438e7e5 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:44.447195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce8d60-127c-41dd-ac9a-04122c125828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.447952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce8d60-127c-41dd-ac9a-04122c125828 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.448268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce8d60-127c-41dd-ac9a-04122c125828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.448641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce8d60-127c-41dd-ac9a-04122c125828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.449991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce8d60-127c-41dd-ac9a-04122c125828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:44.450857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-1f2515d6-bf18-4df5-894c-2bedd47a58d5 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:44.452366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce8d60-127c-41dd-ac9a-04122c125828 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:44.452665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1f2515d6-bf18-4df5-894c-2bedd47a58d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:44.470888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 819/1638] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:44.470888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce8d60-127c-41dd-ac9a-04122c125828 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.490540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce8d60-127c-41dd-ac9a-04122c125828 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:44.497061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0a268-5f7e-470d-b7c9-afea9361e329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.498019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0a268-5f7e-470d-b7c9-afea9361e329 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.498557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0a268-5f7e-470d-b7c9-afea9361e329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.499603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0a268-5f7e-470d-b7c9-afea9361e329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.500439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0a268-5f7e-470d-b7c9-afea9361e329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:44.503312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0a268-5f7e-470d-b7c9-afea9361e329 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:44.519979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 820/1639] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:44.521208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0a268-5f7e-470d-b7c9-afea9361e329 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.613060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0a268-5f7e-470d-b7c9-afea9361e329 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:44.617497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31492985-17c9-4f0f-abfc-44f33bdea6b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.618237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31492985-17c9-4f0f-abfc-44f33bdea6b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.619356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31492985-17c9-4f0f-abfc-44f33bdea6b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.619626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31492985-17c9-4f0f-abfc-44f33bdea6b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.627642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 820/1640] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:21:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:44.637311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2515d6-bf18-4df5-894c-2bedd47a58d5 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.662398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2515d6-bf18-4df5-894c-2bedd47a58d5 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:44.677105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b9ba9-9a64-4b4c-b679-f83742c84a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.677834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b9ba9-9a64-4b4c-b679-f83742c84a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.678195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b9ba9-9a64-4b4c-b679-f83742c84a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.678463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b9ba9-9a64-4b4c-b679-f83742c84a41 admin admin] PATH_INFO: `/v3/projects/5771fcd3dad94c8ca563ccd6162feeb0/users/beceee078d3b479ebe96c755831452d2/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.716042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b9ba9-9a64-4b4c-b679-f83742c84a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5771fcd3dad94c8ca563ccd6162feeb0, user_id=beceee078d3b479ebe96c755831452d2, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:44.720223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31492985-17c9-4f0f-abfc-44f33bdea6b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:44.721420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b9ba9-9a64-4b4c-b679-f83742c84a41 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:44.722716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31492985-17c9-4f0f-abfc-44f33bdea6b1 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:44.731827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:44.744786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 821/1641] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:44.761160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31492985-17c9-4f0f-abfc-44f33bdea6b1 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.768941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 821/1642] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:44 2022] PUT /identity/v3/projects/5771fcd3dad94c8ca563ccd6162feeb0/users/beceee078d3b479ebe96c755831452d2/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:44.774642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b9ba9-9a64-4b4c-b679-f83742c84a41 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.786636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31492985-17c9-4f0f-abfc-44f33bdea6b1 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:44.931452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cd1b7-2ce4-41f9-8627-96f1629804bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.932514 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cd1b7-2ce4-41f9-8627-96f1629804bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.933287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cd1b7-2ce4-41f9-8627-96f1629804bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.933737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cd1b7-2ce4-41f9-8627-96f1629804bc admin admin] PATH_INFO: `/v3/projects/aaff62847f4d4c82b76ba8cbf27bb02c/users/fbabef64c6344dcaad0a04b482df8139/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:44.937835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cd1b7-2ce4-41f9-8627-96f1629804bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaff62847f4d4c82b76ba8cbf27bb02c, user_id=fbabef64c6344dcaad0a04b482df8139, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:44.943526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cd1b7-2ce4-41f9-8627-96f1629804bc admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:44.947625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b9ba9-9a64-4b4c-b679-f83742c84a41 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:44.953178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:44.966577 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 822/1643] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:44 2022] PUT /identity/v3/projects/aaff62847f4d4c82b76ba8cbf27bb02c/users/fbabef64c6344dcaad0a04b482df8139/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:44.977984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cd1b7-2ce4-41f9-8627-96f1629804bc admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:44.978177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85a74a-43d3-4aef-a1e9-4c547b280502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:44.980005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85a74a-43d3-4aef-a1e9-4c547b280502 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:44.980310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85a74a-43d3-4aef-a1e9-4c547b280502 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:44.980580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85a74a-43d3-4aef-a1e9-4c547b280502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:45.106884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cd1b7-2ce4-41f9-8627-96f1629804bc admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:45.112165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2135292-995b-4dde-bb11-d37e71e49a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:45.112783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2135292-995b-4dde-bb11-d37e71e49a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:45.113500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2135292-995b-4dde-bb11-d37e71e49a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:45.113500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2135292-995b-4dde-bb11-d37e71e49a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:45.114044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2135292-995b-4dde-bb11-d37e71e49a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:45.114254 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85a74a-43d3-4aef-a1e9-4c547b280502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:45.115681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2135292-995b-4dde-bb11-d37e71e49a82 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:45.115933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85a74a-43d3-4aef-a1e9-4c547b280502 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:45.131338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 823/1644] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:45.132769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8471d-b8d9-459a-887c-5442e618441a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:45.133056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8471d-b8d9-459a-887c-5442e618441a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:45.133331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8471d-b8d9-459a-887c-5442e618441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:45.144378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 822/1645] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:45.145811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85a74a-43d3-4aef-a1e9-4c547b280502 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:45.164448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85a74a-43d3-4aef-a1e9-4c547b280502 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:45.174487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-40d8471d-b8d9-459a-887c-5442e618441a None None] MFA Rules not processed for user `fbabef64c6344dcaad0a04b482df8139`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:45.175724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-40d8471d-b8d9-459a-887c-5442e618441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:45.280997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a960d-6072-44fc-8e76-6284adc826a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:45.282008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a960d-6072-44fc-8e76-6284adc826a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:45.282683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a960d-6072-44fc-8e76-6284adc826a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:45.283697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a960d-6072-44fc-8e76-6284adc826a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:45.357543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 824/1646] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:45.359458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8471d-b8d9-459a-887c-5442e618441a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:45.397696 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8471d-b8d9-459a-887c-5442e618441a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:45.410644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b146282-91a4-4983-bbce-bcbe4e56c17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:45.412451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b146282-91a4-4983-bbce-bcbe4e56c17f admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:45.412960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b146282-91a4-4983-bbce-bcbe4e56c17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:45.413234 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b146282-91a4-4983-bbce-bcbe4e56c17f admin admin] PATH_INFO: `/v3/projects/5771fcd3dad94c8ca563ccd6162feeb0/users/beceee078d3b479ebe96c755831452d2/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:45.419058 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b146282-91a4-4983-bbce-bcbe4e56c17f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5771fcd3dad94c8ca563ccd6162feeb0, user_id=beceee078d3b479ebe96c755831452d2, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:45.427417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b146282-91a4-4983-bbce-bcbe4e56c17f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:45.439424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:45.455006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 825/1647] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:45 2022] PUT /identity/v3/projects/5771fcd3dad94c8ca563ccd6162feeb0/users/beceee078d3b479ebe96c755831452d2/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:45.465645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937af908-4be5-42a6-963e-0c1e1801b05a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:45.465944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937af908-4be5-42a6-963e-0c1e1801b05a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:45.466203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937af908-4be5-42a6-963e-0c1e1801b05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:45.476479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a960d-6072-44fc-8e76-6284adc826a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:45.478692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a960d-6072-44fc-8e76-6284adc826a6 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:45.489874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 823/1648] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:45.527794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-937af908-4be5-42a6-963e-0c1e1801b05a None None] MFA Rules not processed for user `beceee078d3b479ebe96c755831452d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:45.529039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-937af908-4be5-42a6-963e-0c1e1801b05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:45.657691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:45.710009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 826/1649] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:45 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:21:45.776448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a960d-6072-44fc-8e76-6284adc826a6 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:45.788349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a960d-6072-44fc-8e76-6284adc826a6 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:45.828098 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937af908-4be5-42a6-963e-0c1e1801b05a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:45.930912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c67915-be42-4555-ab28-a9d0068a4ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:45.934043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c67915-be42-4555-ab28-a9d0068a4ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:45.934043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c67915-be42-4555-ab28-a9d0068a4ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:45.934259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c67915-be42-4555-ab28-a9d0068a4ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:45.963343 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937af908-4be5-42a6-963e-0c1e1801b05a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:45.978926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307f6b-87a9-4f0d-bf41-896c015a4f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:45.979731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307f6b-87a9-4f0d-bf41-896c015a4f72 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:45.980103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307f6b-87a9-4f0d-bf41-896c015a4f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:45.980464 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6307f6b-87a9-4f0d-bf41-896c015a4f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:46.006272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6307f6b-87a9-4f0d-bf41-896c015a4f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:46.007828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6307f6b-87a9-4f0d-bf41-896c015a4f72 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:46.027949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 827/1650] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:45 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:21:46.102071 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c67915-be42-4555-ab28-a9d0068a4ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:46.112416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c67915-be42-4555-ab28-a9d0068a4ff6 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:46.120752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 824/1651] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:46.171737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307f6b-87a9-4f0d-bf41-896c015a4f72 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:46.378659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6307f6b-87a9-4f0d-bf41-896c015a4f72 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:46.392853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:46.394407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:46.394891 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:46.397116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] PATH_INFO: `/v3/users/f87d19eadffc4ab6ad22d95f865bce27` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:46.402674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87d19eadffc4ab6ad22d95f865bce27)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:46.405834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: WARNING py.warnings [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:21:46.405834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: warnings.warn(msg) Oct 05 20:21:46.405834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: Oct 05 20:21:46.410212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:46.569427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87d19eadffc4ab6ad22d95f865bce27'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:46.570625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87d19eadffc4ab6ad22d95f865bce27'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:46.596529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87d19eadffc4ab6ad22d95f865bce27'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:46.597603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87d19eadffc4ab6ad22d95f865bce27'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:46.607627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87d19eadffc4ab6ad22d95f865bce27'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:46.658979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 828/1652] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:21:46 2022] DELETE /identity/v3/users/f87d19eadffc4ab6ad22d95f865bce27 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:21:46.663519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c044a04-5046-40f0-a9d9-b9624f26d612 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:46.664163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c044a04-5046-40f0-a9d9-b9624f26d612 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:46.664163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c044a04-5046-40f0-a9d9-b9624f26d612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:46.701219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4c044a04-5046-40f0-a9d9-b9624f26d612 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:46.702526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4c044a04-5046-40f0-a9d9-b9624f26d612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:46.834121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 825/1653] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:46.852180 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:46.853686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:46.885281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:46.997299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a122ac9-1230-4e18-8589-95ff1263ab5f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:47.007959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc82c7-edd8-4776-967d-b618f0a58259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:47.008602 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc82c7-edd8-4776-967d-b618f0a58259 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:47.008880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc82c7-edd8-4776-967d-b618f0a58259 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:47.009151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc82c7-edd8-4776-967d-b618f0a58259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:47.034049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc82c7-edd8-4776-967d-b618f0a58259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:47.037283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc82c7-edd8-4776-967d-b618f0a58259 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:47.070795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 829/1654] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:48.693751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c044a04-5046-40f0-a9d9-b9624f26d612 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:48.695371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4c044a04-5046-40f0-a9d9-b9624f26d612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:48.716782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c044a04-5046-40f0-a9d9-b9624f26d612 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:48.730988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:48.731817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:48.732197 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:48.732568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] PATH_INFO: `/v3/projects/e43dabe5f2b34bf29e56413ca0e21886` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:48.735002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43dabe5f2b34bf29e56413ca0e21886)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:48.736267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: WARNING py.warnings [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 20:21:48.736267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: warnings.warn(msg) Oct 05 20:21:48.736267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: Oct 05 20:21:48.737722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:48.799786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:48.842961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43dabe5f2b34bf29e56413ca0e21886', 'request_id': 'req-30533c66-f64a-4a9d-acfd-5e3d79f376f7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:48.863514 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43dabe5f2b34bf29e56413ca0e21886', 'request_id': 'req-30533c66-f64a-4a9d-acfd-5e3d79f376f7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:48.891764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43dabe5f2b34bf29e56413ca0e21886', 'request_id': 'req-30533c66-f64a-4a9d-acfd-5e3d79f376f7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:48.911217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] The token cache is being invalidate because project e43dabe5f2b34bf29e56413ca0e21886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:21:48.911675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:48.913417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:48.918209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 826/1655] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:21:48 2022] DELETE /identity/v3/projects/e43dabe5f2b34bf29e56413ca0e21886 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:48.919776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc82c7-edd8-4776-967d-b618f0a58259 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:48.931418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc82c7-edd8-4776-967d-b618f0a58259 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:49.018107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:49.019263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:49.019725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:49.020124 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] PATH_INFO: `/v3/users/fbabef64c6344dcaad0a04b482df8139` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:49.021898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbabef64c6344dcaad0a04b482df8139)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:49.024141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:49.121820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbabef64c6344dcaad0a04b482df8139'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.121975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbabef64c6344dcaad0a04b482df8139'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.137710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbabef64c6344dcaad0a04b482df8139'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.139907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbabef64c6344dcaad0a04b482df8139'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.163149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b5589942-2280-442d-a94e-3be1d3aa90e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbabef64c6344dcaad0a04b482df8139'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.197911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:49.203329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:49.237798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 830/1656] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:21:48 2022] DELETE /identity/v3/users/fbabef64c6344dcaad0a04b482df8139 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:49.238579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30533c66-f64a-4a9d-acfd-5e3d79f376f7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:49.393342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:49.393342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:49.394604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:49.395336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] PATH_INFO: `/v3/users/cbaa416043eb4b4f9b628625f0cddd6b` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:49.401308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbaa416043eb4b4f9b628625f0cddd6b)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:49.410635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:49.616897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbaa416043eb4b4f9b628625f0cddd6b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.618420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbaa416043eb4b4f9b628625f0cddd6b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.647808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbaa416043eb4b4f9b628625f0cddd6b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.673750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbaa416043eb4b4f9b628625f0cddd6b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.674404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbaa416043eb4b4f9b628625f0cddd6b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:49.725185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 827/1657] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:21:49 2022] DELETE /identity/v3/users/cbaa416043eb4b4f9b628625f0cddd6b => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:49.728913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e59d3c-8bc6-4b37-9f0b-d8560d9fce77 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:49.729552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e59d3c-8bc6-4b37-9f0b-d8560d9fce77 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:49.729840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e59d3c-8bc6-4b37-9f0b-d8560d9fce77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:49.780702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-88e59d3c-8bc6-4b37-9f0b-d8560d9fce77 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:49.785252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-88e59d3c-8bc6-4b37-9f0b-d8560d9fce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:49.871595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:49.975192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 831/1658] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:50.145004 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:50.146222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:50.300130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:50.346372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346a0fd-183a-40bd-b43e-36dca97392b7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:50.364109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e35273-9759-460b-837a-7677945fb798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:50.364766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e35273-9759-460b-837a-7677945fb798 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:50.365120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e35273-9759-460b-837a-7677945fb798 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:50.365307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e35273-9759-460b-837a-7677945fb798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:50.422523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e35273-9759-460b-837a-7677945fb798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:50.432834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e35273-9759-460b-837a-7677945fb798 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:50.474287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 828/1659] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:51.460377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e59d3c-8bc6-4b37-9f0b-d8560d9fce77 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:51.462037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-88e59d3c-8bc6-4b37-9f0b-d8560d9fce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:51.482634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e59d3c-8bc6-4b37-9f0b-d8560d9fce77 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:51.494414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:51.495078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:51.495358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:51.497660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] PATH_INFO: `/v3/projects/aaff62847f4d4c82b76ba8cbf27bb02c` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:51.500773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaff62847f4d4c82b76ba8cbf27bb02c)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:51.502775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:51.630595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaff62847f4d4c82b76ba8cbf27bb02c', 'request_id': 'req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:51.671870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaff62847f4d4c82b76ba8cbf27bb02c', 'request_id': 'req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:51.693713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaff62847f4d4c82b76ba8cbf27bb02c', 'request_id': 'req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:51.747629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] The token cache is being invalidate because project aaff62847f4d4c82b76ba8cbf27bb02c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:21:51.748089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:51.749090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-eeefa0fd-68de-48ce-8d73-1bfb11b31ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:51.754513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 832/1660] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:21:51 2022] DELETE /identity/v3/projects/aaff62847f4d4c82b76ba8cbf27bb02c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:51.755742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401b954-306e-4f34-ac9f-1b0ef61f676a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:51.756566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401b954-306e-4f34-ac9f-1b0ef61f676a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:51.756962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401b954-306e-4f34-ac9f-1b0ef61f676a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:51.800980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9401b954-306e-4f34-ac9f-1b0ef61f676a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:51.806289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9401b954-306e-4f34-ac9f-1b0ef61f676a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:51.965830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 829/1661] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:51.975868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f302ff5-9770-4ec6-a248-7bfb9161369f None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:51.976156 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f302ff5-9770-4ec6-a248-7bfb9161369f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:51.976435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f302ff5-9770-4ec6-a248-7bfb9161369f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:52.015527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-4f302ff5-9770-4ec6-a248-7bfb9161369f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:52.016967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4f302ff5-9770-4ec6-a248-7bfb9161369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:52.042525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 833/1662] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:52.043528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401b954-306e-4f34-ac9f-1b0ef61f676a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:52.056192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401b954-306e-4f34-ac9f-1b0ef61f676a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:52.071000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45118bd2-9237-449a-81a7-f04bf04c1a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:52.071638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45118bd2-9237-449a-81a7-f04bf04c1a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:52.071915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45118bd2-9237-449a-81a7-f04bf04c1a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:52.072187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45118bd2-9237-449a-81a7-f04bf04c1a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:52.072843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45118bd2-9237-449a-81a7-f04bf04c1a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:52.073144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45118bd2-9237-449a-81a7-f04bf04c1a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:52.074969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45118bd2-9237-449a-81a7-f04bf04c1a46 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:52.107239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 830/1663] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:21:52.112790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f302ff5-9770-4ec6-a248-7bfb9161369f None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:52.129744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f302ff5-9770-4ec6-a248-7bfb9161369f None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:52.134741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07d1c0-d281-476e-8cd8-9fa24440c95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:52.135372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07d1c0-d281-476e-8cd8-9fa24440c95c admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:52.135650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07d1c0-d281-476e-8cd8-9fa24440c95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:52.137686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07d1c0-d281-476e-8cd8-9fa24440c95c admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:52.138360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07d1c0-d281-476e-8cd8-9fa24440c95c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:52.145709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07d1c0-d281-476e-8cd8-9fa24440c95c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:52.223128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 834/1664] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:52 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:52.223544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45118bd2-9237-449a-81a7-f04bf04c1a46 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:52.240033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45118bd2-9237-449a-81a7-f04bf04c1a46 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:52.365812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe619-b409-4669-85cd-369c6eba75be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:52.366447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe619-b409-4669-85cd-369c6eba75be admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:52.366734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe619-b409-4669-85cd-369c6eba75be admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:52.367003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe619-b409-4669-85cd-369c6eba75be admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:52.367793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbe619-b409-4669-85cd-369c6eba75be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:52.369752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbe619-b409-4669-85cd-369c6eba75be admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:52.414046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 831/1665] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:52 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:52.420845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07d1c0-d281-476e-8cd8-9fa24440c95c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:52.437284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07d1c0-d281-476e-8cd8-9fa24440c95c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:52.450313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a336ac-3c40-4a0c-a307-05fc0ada04be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:52.450969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a336ac-3c40-4a0c-a307-05fc0ada04be admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:52.453150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a336ac-3c40-4a0c-a307-05fc0ada04be admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:52.453455 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a336ac-3c40-4a0c-a307-05fc0ada04be admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:52.454177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a336ac-3c40-4a0c-a307-05fc0ada04be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:52.472554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a336ac-3c40-4a0c-a307-05fc0ada04be admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:52.510399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 835/1666] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:52.514089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe619-b409-4669-85cd-369c6eba75be admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:52.531279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe619-b409-4669-85cd-369c6eba75be admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:52.551850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120af348-29be-4f1a-b91b-7200f94f0527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:52.552490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120af348-29be-4f1a-b91b-7200f94f0527 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:52.552765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120af348-29be-4f1a-b91b-7200f94f0527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:52.553038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120af348-29be-4f1a-b91b-7200f94f0527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:52.553817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120af348-29be-4f1a-b91b-7200f94f0527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:52.555327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120af348-29be-4f1a-b91b-7200f94f0527 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:52.585010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 832/1667] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:52.589506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a336ac-3c40-4a0c-a307-05fc0ada04be admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:52.607112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a336ac-3c40-4a0c-a307-05fc0ada04be admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:52.621649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d9ddb-be51-4f57-95f6-834395085505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:52.622346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d9ddb-be51-4f57-95f6-834395085505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:52.622625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d9ddb-be51-4f57-95f6-834395085505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:52.622974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d9ddb-be51-4f57-95f6-834395085505 admin admin] PATH_INFO: `/v3/projects/74cb11b256ac497da3e8996bc8887351/users/cf9a836a8ca64295a4289654378c2db6/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:52.658201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d9ddb-be51-4f57-95f6-834395085505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74cb11b256ac497da3e8996bc8887351, user_id=cf9a836a8ca64295a4289654378c2db6, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:52.661366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d9ddb-be51-4f57-95f6-834395085505 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:52.695426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 836/1668] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:52 2022] PUT /identity/v3/projects/74cb11b256ac497da3e8996bc8887351/users/cf9a836a8ca64295a4289654378c2db6/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:52.697291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f4ed0-14a4-45dc-9331-94ebebc9077e None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:52.697603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f4ed0-14a4-45dc-9331-94ebebc9077e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:52.698279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f4ed0-14a4-45dc-9331-94ebebc9077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:52.754461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-da8f4ed0-14a4-45dc-9331-94ebebc9077e None None] MFA Rules not processed for user `cf9a836a8ca64295a4289654378c2db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:52.755906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-da8f4ed0-14a4-45dc-9331-94ebebc9077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:52.897117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:52.961735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 833/1669] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:52.967967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d9ddb-be51-4f57-95f6-834395085505 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:53.073712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f4ed0-14a4-45dc-9331-94ebebc9077e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:53.075756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-da8f4ed0-14a4-45dc-9331-94ebebc9077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:53.124683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:53.143598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d9ddb-be51-4f57-95f6-834395085505 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:53.155995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b7368-c923-4109-8075-def4984994bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:53.156995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b7368-c923-4109-8075-def4984994bb admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:53.157279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b7368-c923-4109-8075-def4984994bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:53.157790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b7368-c923-4109-8075-def4984994bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:53.158454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b7368-c923-4109-8075-def4984994bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:53.160434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b7368-c923-4109-8075-def4984994bb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:53.231864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 837/1670] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:52 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:53.236032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b7368-c923-4109-8075-def4984994bb admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:53.246713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f4ed0-14a4-45dc-9331-94ebebc9077e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:53.366178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b7368-c923-4109-8075-def4984994bb admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:53.372837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:53.374616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:53.374902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:53.375521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] PATH_INFO: `/v3/projects/46c4e531a28a4944a2beab1d8751a07b` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:53.377811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46c4e531a28a4944a2beab1d8751a07b)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:53.379765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:53.382119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f22b67-dc9a-420b-bce5-0209241a410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:53.382733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f22b67-dc9a-420b-bce5-0209241a410d admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:53.383011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f22b67-dc9a-420b-bce5-0209241a410d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:53.383278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f22b67-dc9a-420b-bce5-0209241a410d admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:53.383917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f22b67-dc9a-420b-bce5-0209241a410d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:53.385973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f22b67-dc9a-420b-bce5-0209241a410d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:53.444869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 838/1671] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:53 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:53.448326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f22b67-dc9a-420b-bce5-0209241a410d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:53.470572 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f22b67-dc9a-420b-bce5-0209241a410d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:53.486608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350a958-ab13-48c4-b377-03b82d6b486f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:53.487295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350a958-ab13-48c4-b377-03b82d6b486f admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:53.487823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350a958-ab13-48c4-b377-03b82d6b486f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:53.488379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350a958-ab13-48c4-b377-03b82d6b486f admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:53.489610 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350a958-ab13-48c4-b377-03b82d6b486f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:53.491681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350a958-ab13-48c4-b377-03b82d6b486f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:53.507477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46c4e531a28a4944a2beab1d8751a07b', 'request_id': 'req-ea4dee56-5326-4d6d-9541-5592db6809c9'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.523808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 839/1672] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:21:53.527222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350a958-ab13-48c4-b377-03b82d6b486f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:53.535894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46c4e531a28a4944a2beab1d8751a07b', 'request_id': 'req-ea4dee56-5326-4d6d-9541-5592db6809c9'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.552359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350a958-ab13-48c4-b377-03b82d6b486f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:53.560830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46c4e531a28a4944a2beab1d8751a07b', 'request_id': 'req-ea4dee56-5326-4d6d-9541-5592db6809c9'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.581003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] The token cache is being invalidate because project 46c4e531a28a4944a2beab1d8751a07b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:21:53.581288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.586777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.591412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 834/1673] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:21:53 2022] DELETE /identity/v3/projects/46c4e531a28a4944a2beab1d8751a07b => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:53.593011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:53.594516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:53.641503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:53.721280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dee56-5326-4d6d-9541-5592db6809c9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:53.731655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:53.732322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:53.732600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:53.733440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] PATH_INFO: `/v3/users/beceee078d3b479ebe96c755831452d2` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:53.734619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beceee078d3b479ebe96c755831452d2)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:53.736557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:53.783303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee595a1e-dd53-4674-9345-c46e7bc59d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:53.784702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee595a1e-dd53-4674-9345-c46e7bc59d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:53.785004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee595a1e-dd53-4674-9345-c46e7bc59d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:53.785277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee595a1e-dd53-4674-9345-c46e7bc59d43 admin admin] PATH_INFO: `/v3/projects/17fd8aeb882347af98589aef1b293e6a/users/e1ae8750ba654dee94fa60e4296c892c/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:53.839251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beceee078d3b479ebe96c755831452d2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.840024 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beceee078d3b479ebe96c755831452d2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.855797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beceee078d3b479ebe96c755831452d2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.857028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee595a1e-dd53-4674-9345-c46e7bc59d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fd8aeb882347af98589aef1b293e6a, user_id=e1ae8750ba654dee94fa60e4296c892c, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:53.876782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beceee078d3b479ebe96c755831452d2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.877443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54fbc712-233e-419c-8261-771cd6091a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beceee078d3b479ebe96c755831452d2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:53.880202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee595a1e-dd53-4674-9345-c46e7bc59d43 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:53.912571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 840/1674] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:53 2022] PUT /identity/v3/projects/17fd8aeb882347af98589aef1b293e6a/users/e1ae8750ba654dee94fa60e4296c892c/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:53.917259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee595a1e-dd53-4674-9345-c46e7bc59d43 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:53.918862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 835/1675] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:21:53 2022] DELETE /identity/v3/users/beceee078d3b479ebe96c755831452d2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:53.922874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ee595a1e-dd53-4674-9345-c46e7bc59d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:54.090777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:54.123380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee595a1e-dd53-4674-9345-c46e7bc59d43 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:54.219399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ded19-7d02-4d05-a364-8344670eb2bc None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:54.219686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ded19-7d02-4d05-a364-8344670eb2bc None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:54.219968 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ded19-7d02-4d05-a364-8344670eb2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:54.283288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc3861-6196-49ea-94b3-a00697ae6272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:54.284398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc3861-6196-49ea-94b3-a00697ae6272 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:54.284849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc3861-6196-49ea-94b3-a00697ae6272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:54.286725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc3861-6196-49ea-94b3-a00697ae6272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:54.288038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc3861-6196-49ea-94b3-a00697ae6272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:54.290345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-fe6ded19-7d02-4d05-a364-8344670eb2bc None None] MFA Rules not processed for user `1301a78c1a684c94acb9f26b56f39f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:54.291096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc3861-6196-49ea-94b3-a00697ae6272 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:54.291693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fe6ded19-7d02-4d05-a364-8344670eb2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:54.307839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 841/1676] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:54.317483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc3861-6196-49ea-94b3-a00697ae6272 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:54.343333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc3861-6196-49ea-94b3-a00697ae6272 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:54.356149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badfe7b-e139-4e91-95c3-68d01c9e30b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:54.357030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badfe7b-e139-4e91-95c3-68d01c9e30b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:54.359025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badfe7b-e139-4e91-95c3-68d01c9e30b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:54.359529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badfe7b-e139-4e91-95c3-68d01c9e30b7 admin admin] PATH_INFO: `/v3/projects/17fd8aeb882347af98589aef1b293e6a/users/e1ae8750ba654dee94fa60e4296c892c/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:54.364720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3badfe7b-e139-4e91-95c3-68d01c9e30b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fd8aeb882347af98589aef1b293e6a, user_id=e1ae8750ba654dee94fa60e4296c892c, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:54.369163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3badfe7b-e139-4e91-95c3-68d01c9e30b7 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:54.403010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 842/1677] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:54 2022] PUT /identity/v3/projects/17fd8aeb882347af98589aef1b293e6a/users/e1ae8750ba654dee94fa60e4296c892c/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:54.413547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b356e9-43e4-4328-9ade-e4ac2d952082 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:54.414168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b356e9-43e4-4328-9ade-e4ac2d952082 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:54.414586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b356e9-43e4-4328-9ade-e4ac2d952082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:54.467189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 836/1678] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:21:54.477748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dbed0-17cf-4d1f-b73d-c144b94e03f7 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:54.478037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dbed0-17cf-4d1f-b73d-c144b94e03f7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:54.478311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dbed0-17cf-4d1f-b73d-c144b94e03f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:54.482052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-80b356e9-43e4-4328-9ade-e4ac2d952082 None None] MFA Rules not processed for user `e1ae8750ba654dee94fa60e4296c892c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:54.488015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-80b356e9-43e4-4328-9ade-e4ac2d952082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:54.516076 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-890dbed0-17cf-4d1f-b73d-c144b94e03f7 None None] MFA Rules not processed for user `1301a78c1a684c94acb9f26b56f39f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:54.519159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-890dbed0-17cf-4d1f-b73d-c144b94e03f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:54.597625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:54.688907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 843/1679] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:21:54.723135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b356e9-43e4-4328-9ade-e4ac2d952082 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:54.725699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-80b356e9-43e4-4328-9ade-e4ac2d952082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:54.753092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 837/1680] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:54.755855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dbed0-17cf-4d1f-b73d-c144b94e03f7 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:54.757615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-890dbed0-17cf-4d1f-b73d-c144b94e03f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:54.969904 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b356e9-43e4-4328-9ade-e4ac2d952082 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:54.984317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0653c6-8177-4041-82c6-40e6267a8c03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:54.984958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0653c6-8177-4041-82c6-40e6267a8c03 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:54.985235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0653c6-8177-4041-82c6-40e6267a8c03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:54.986966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0653c6-8177-4041-82c6-40e6267a8c03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:55.008975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0653c6-8177-4041-82c6-40e6267a8c03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:55.010917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0653c6-8177-4041-82c6-40e6267a8c03 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:55.020482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:55.073120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dbed0-17cf-4d1f-b73d-c144b94e03f7 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:55.080893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 844/1681] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:55.084533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1a5e6-bf4a-4852-b8e6-cb73fb8bea07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:55.085166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a5e6-bf4a-4852-b8e6-cb73fb8bea07 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:55.085449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a5e6-bf4a-4852-b8e6-cb73fb8bea07 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:55.085751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a5e6-bf4a-4852-b8e6-cb73fb8bea07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:55.130827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1a5e6-bf4a-4852-b8e6-cb73fb8bea07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:55.132446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1a5e6-bf4a-4852-b8e6-cb73fb8bea07 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:55.159513 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 838/1682] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:55.395618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0653c6-8177-4041-82c6-40e6267a8c03 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:55.406564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0653c6-8177-4041-82c6-40e6267a8c03 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:55.431341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf84173-4792-4cd6-996f-35773410cb85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:55.432146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf84173-4792-4cd6-996f-35773410cb85 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:55.432504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf84173-4792-4cd6-996f-35773410cb85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:55.432785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf84173-4792-4cd6-996f-35773410cb85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:55.439334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-dcf84173-4792-4cd6-996f-35773410cb85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:55.580378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf84173-4792-4cd6-996f-35773410cb85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:55.581902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf84173-4792-4cd6-996f-35773410cb85 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:55.614032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 845/1683] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:55.803923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1a5e6-bf4a-4852-b8e6-cb73fb8bea07 service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:55.805532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-60b1a5e6-bf4a-4852-b8e6-cb73fb8bea07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:56.098833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1a5e6-bf4a-4852-b8e6-cb73fb8bea07 service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:56.118708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:56.119374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:56.119653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:56.119924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] PATH_INFO: `/v3/users/04c46460cf0c478fbf21499f6cceaf65` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:56.137558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c46460cf0c478fbf21499f6cceaf65)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:56.137845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:56.280850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c46460cf0c478fbf21499f6cceaf65'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:56.280850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c46460cf0c478fbf21499f6cceaf65'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:56.302814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c46460cf0c478fbf21499f6cceaf65'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:56.314783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:56.334634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c46460cf0c478fbf21499f6cceaf65'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:56.336020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c46460cf0c478fbf21499f6cceaf65'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:56.403678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b87c0-9840-4dbb-a472-739884690263 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:56.405108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 839/1684] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:21:55 2022] DELETE /identity/v3/users/04c46460cf0c478fbf21499f6cceaf65 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:56.405878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b87c0-9840-4dbb-a472-739884690263 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:56.407548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b87c0-9840-4dbb-a472-739884690263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:56.455871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-2e1b87c0-9840-4dbb-a472-739884690263 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:56.458642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2e1b87c0-9840-4dbb-a472-739884690263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:56.638513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 846/1685] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:56.672391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:56.675181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:56.882478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79d938-2d4e-48a6-b4bb-b5b75ef81660 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:56.882478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79d938-2d4e-48a6-b4bb-b5b75ef81660 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:56.882478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79d938-2d4e-48a6-b4bb-b5b75ef81660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:56.983786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619b102-0ff8-4758-9ee5-b03c8c9bc0fa admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:56.984735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-2e79d938-2d4e-48a6-b4bb-b5b75ef81660 None None] MFA Rules not processed for user `ff9b14ab630d4a928614c60e6aea667c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:56.986136 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2e79d938-2d4e-48a6-b4bb-b5b75ef81660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:57.003078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ded6e-08f0-4361-ac8f-9c9a7a370bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:57.004122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ded6e-08f0-4361-ac8f-9c9a7a370bfe service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:57.004627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ded6e-08f0-4361-ac8f-9c9a7a370bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:57.005235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ded6e-08f0-4361-ac8f-9c9a7a370bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:57.025870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ded6e-08f0-4361-ac8f-9c9a7a370bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:57.048107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ded6e-08f0-4361-ac8f-9c9a7a370bfe service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:57.081691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 840/1686] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:57.158811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 847/1687] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:57.166714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ded6e-08f0-4361-ac8f-9c9a7a370bfe service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:57.168780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-929ded6e-08f0-4361-ac8f-9c9a7a370bfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:57.179715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79d938-2d4e-48a6-b4bb-b5b75ef81660 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:57.181120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2e79d938-2d4e-48a6-b4bb-b5b75ef81660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:57.204480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ded6e-08f0-4361-ac8f-9c9a7a370bfe service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:57.224837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:57.227409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:57.229816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:57.230222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] PATH_INFO: `/v3/projects/5771fcd3dad94c8ca563ccd6162feeb0` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:57.233230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5771fcd3dad94c8ca563ccd6162feeb0)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:57.246329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:57.271318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:57.427042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79d938-2d4e-48a6-b4bb-b5b75ef81660 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:57.431517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5771fcd3dad94c8ca563ccd6162feeb0', 'request_id': 'req-be99993c-7310-4a57-bd37-a320d616d03d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:57.451062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5771fcd3dad94c8ca563ccd6162feeb0', 'request_id': 'req-be99993c-7310-4a57-bd37-a320d616d03d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:57.465632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5771fcd3dad94c8ca563ccd6162feeb0', 'request_id': 'req-be99993c-7310-4a57-bd37-a320d616d03d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:57.496596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] The token cache is being invalidate because project 5771fcd3dad94c8ca563ccd6162feeb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:21:57.496913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:57.499521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-be99993c-7310-4a57-bd37-a320d616d03d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:57.503973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 841/1688] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:21:57 2022] DELETE /identity/v3/projects/5771fcd3dad94c8ca563ccd6162feeb0 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:57.507780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d083389-771e-47f5-85a1-aa6361cf931d None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:57.509193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d083389-771e-47f5-85a1-aa6361cf931d None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:57.509495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d083389-771e-47f5-85a1-aa6361cf931d None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:57.514697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 842/1689] 10.222.0.23 () {54 vars in 912 bytes} [Wed Oct 5 20:21:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:21:57.516085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b530d-5375-4113-8390-57477dbab6ce None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:57.516370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b530d-5375-4113-8390-57477dbab6ce None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:57.518686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b530d-5375-4113-8390-57477dbab6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:57.577742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-969b530d-5375-4113-8390-57477dbab6ce None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:57.581804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-969b530d-5375-4113-8390-57477dbab6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:57.611361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829611e-a59d-4d5c-9636-0bcb78cc5649 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:57.611992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829611e-a59d-4d5c-9636-0bcb78cc5649 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:57.612268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829611e-a59d-4d5c-9636-0bcb78cc5649 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:57.613294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829611e-a59d-4d5c-9636-0bcb78cc5649 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:57.615937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7829611e-a59d-4d5c-9636-0bcb78cc5649 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:57.745991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 843/1690] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:57.747578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad6cb7-2955-4277-8e1e-30b255462763 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:57.747868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad6cb7-2955-4277-8e1e-30b255462763 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:57.748690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad6cb7-2955-4277-8e1e-30b255462763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:57.775707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:57.789741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829611e-a59d-4d5c-9636-0bcb78cc5649 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:57.794211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829611e-a59d-4d5c-9636-0bcb78cc5649 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:57.825274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 848/1691] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:21:57.826660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95e81d-f4cf-4699-82f9-c39d83d1016e None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:57.826941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95e81d-f4cf-4699-82f9-c39d83d1016e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:57.827208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95e81d-f4cf-4699-82f9-c39d83d1016e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:57.879520 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7aad6cb7-2955-4277-8e1e-30b255462763 None None] MFA Rules not processed for user `8f5a88f263014f15a6f59cfdaa2bb6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:57.880760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7aad6cb7-2955-4277-8e1e-30b255462763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:57.902844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-4b95e81d-f4cf-4699-82f9-c39d83d1016e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:57.924228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4b95e81d-f4cf-4699-82f9-c39d83d1016e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:57.962905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 849/1692] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:57.968409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95e81d-f4cf-4699-82f9-c39d83d1016e None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:58.001909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95e81d-f4cf-4699-82f9-c39d83d1016e None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:58.013746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea38c27-5a49-4223-aa16-a8461491ef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:58.014413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38c27-5a49-4223-aa16-a8461491ef45 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.015135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38c27-5a49-4223-aa16-a8461491ef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.015135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38c27-5a49-4223-aa16-a8461491ef45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.015655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea38c27-5a49-4223-aa16-a8461491ef45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:21:58.015960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea38c27-5a49-4223-aa16-a8461491ef45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:58.019817 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea38c27-5a49-4223-aa16-a8461491ef45 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:58.052992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 850/1693] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:21:58.054592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea38c27-5a49-4223-aa16-a8461491ef45 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:58.073906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea38c27-5a49-4223-aa16-a8461491ef45 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:58.088415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17317a3-6ab0-4f22-be4b-c6811b14cd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:58.089246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17317a3-6ab0-4f22-be4b-c6811b14cd72 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.089323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17317a3-6ab0-4f22-be4b-c6811b14cd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.089652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17317a3-6ab0-4f22-be4b-c6811b14cd72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.090254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17317a3-6ab0-4f22-be4b-c6811b14cd72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:58.092813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17317a3-6ab0-4f22-be4b-c6811b14cd72 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:58.142688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 844/1694] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:21:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:58.212905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 851/1695] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:58 2022] POST /identity/v3/projects => generated 365 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:58.218206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad6cb7-2955-4277-8e1e-30b255462763 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:58.250615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad6cb7-2955-4277-8e1e-30b255462763 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:58.354835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17317a3-6ab0-4f22-be4b-c6811b14cd72 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:58.360087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a17317a3-6ab0-4f22-be4b-c6811b14cd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:58.428702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:58.430189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e09c0-212b-4ea4-9c23-5d11d4ff2121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:58.430903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e09c0-212b-4ea4-9c23-5d11d4ff2121 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.431255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e09c0-212b-4ea4-9c23-5d11d4ff2121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.431599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e09c0-212b-4ea4-9c23-5d11d4ff2121 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.432859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e09c0-212b-4ea4-9c23-5d11d4ff2121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:58.436940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e09c0-212b-4ea4-9c23-5d11d4ff2121 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:58.494930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 845/1696] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:58 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:58.499707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e09c0-212b-4ea4-9c23-5d11d4ff2121 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:58.504764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17317a3-6ab0-4f22-be4b-c6811b14cd72 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:58.508761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e2936-c2f2-4dd6-9e61-3da034c1ae12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:58.509457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e2936-c2f2-4dd6-9e61-3da034c1ae12 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.509849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e2936-c2f2-4dd6-9e61-3da034c1ae12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.510247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e2936-c2f2-4dd6-9e61-3da034c1ae12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.522378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e2936-c2f2-4dd6-9e61-3da034c1ae12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:58.523242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e09c0-212b-4ea4-9c23-5d11d4ff2121 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:58.525423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e2936-c2f2-4dd6-9e61-3da034c1ae12 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:58.540006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dd7ad-4a6d-4526-b603-d10c09bab203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:58.546119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dd7ad-4a6d-4526-b603-d10c09bab203 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.547126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dd7ad-4a6d-4526-b603-d10c09bab203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.548996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dd7ad-4a6d-4526-b603-d10c09bab203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.549417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 852/1697] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:58.555507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dd7ad-4a6d-4526-b603-d10c09bab203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:58.565503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dd7ad-4a6d-4526-b603-d10c09bab203 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:58.612252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e2936-c2f2-4dd6-9e61-3da034c1ae12 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:58.612474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 846/1698] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:58.627640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e2936-c2f2-4dd6-9e61-3da034c1ae12 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:58.639492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53147e0a-af51-455e-b49e-99161ff31a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:58.640241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53147e0a-af51-455e-b49e-99161ff31a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.640344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53147e0a-af51-455e-b49e-99161ff31a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.640573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53147e0a-af51-455e-b49e-99161ff31a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.641348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53147e0a-af51-455e-b49e-99161ff31a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:58.648820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53147e0a-af51-455e-b49e-99161ff31a47 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:58.649786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dd7ad-4a6d-4526-b603-d10c09bab203 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:58.651712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-861dd7ad-4a6d-4526-b603-d10c09bab203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:58.669365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 853/1699] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:58.673155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53147e0a-af51-455e-b49e-99161ff31a47 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:58.678232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dd7ad-4a6d-4526-b603-d10c09bab203 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:58.694839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:58.695456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.695558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53147e0a-af51-455e-b49e-99161ff31a47 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:58.695734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.696057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] PATH_INFO: `/v3/users/3c607b476313496b80aa0d92330597f4` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.700201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c607b476313496b80aa0d92330597f4)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:58.705586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:58.710066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70af0e-084e-4263-b90a-89957305af68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:58.711220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70af0e-084e-4263-b90a-89957305af68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.711220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70af0e-084e-4263-b90a-89957305af68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.711413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70af0e-084e-4263-b90a-89957305af68 admin admin] PATH_INFO: `/v3/projects/caa4eefe582c4b55b199aece42a77ebe/users/8baad395c74d4c7196960010a98fd5f2/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.746515 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70af0e-084e-4263-b90a-89957305af68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa4eefe582c4b55b199aece42a77ebe, user_id=8baad395c74d4c7196960010a98fd5f2, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:58.751674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70af0e-084e-4263-b90a-89957305af68 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:58.795412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 854/1700] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:58 2022] PUT /identity/v3/projects/caa4eefe582c4b55b199aece42a77ebe/users/8baad395c74d4c7196960010a98fd5f2/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:58.807873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecedc6d-bc4d-401c-891d-55a6c17ff2c2 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.808425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecedc6d-bc4d-401c-891d-55a6c17ff2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.808893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecedc6d-bc4d-401c-891d-55a6c17ff2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.851595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c607b476313496b80aa0d92330597f4'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:58.854219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c607b476313496b80aa0d92330597f4'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:58.880031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c607b476313496b80aa0d92330597f4'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:58.898799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c607b476313496b80aa0d92330597f4'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:58.899466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c98f572-ac42-44d1-9178-8a1da5b10df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c607b476313496b80aa0d92330597f4'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:58.901661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7ecedc6d-bc4d-401c-891d-55a6c17ff2c2 None None] MFA Rules not processed for user `8baad395c74d4c7196960010a98fd5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:58.909809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7ecedc6d-bc4d-401c-891d-55a6c17ff2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:58.913844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:58.940439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 847/1701] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:21:58 2022] DELETE /identity/v3/users/3c607b476313496b80aa0d92330597f4 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:58.945019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b9288-1a17-4059-b813-f3ebee9fe393 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:58.945444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b9288-1a17-4059-b813-f3ebee9fe393 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:58.945590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b9288-1a17-4059-b813-f3ebee9fe393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:58.989479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-575b9288-1a17-4059-b813-f3ebee9fe393 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:21:58.993628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-575b9288-1a17-4059-b813-f3ebee9fe393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:59.098294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 855/1702] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:21:59.099496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecedc6d-bc4d-401c-891d-55a6c17ff2c2 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:59.105412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7ecedc6d-bc4d-401c-891d-55a6c17ff2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:59.207452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 848/1703] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:21:59.241928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b9288-1a17-4059-b813-f3ebee9fe393 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:59.244556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-575b9288-1a17-4059-b813-f3ebee9fe393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:59.292921 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecedc6d-bc4d-401c-891d-55a6c17ff2c2 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:59.304963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5c6f0-95b2-4092-a505-c29ab25c8b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:59.306081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5c6f0-95b2-4092-a505-c29ab25c8b13 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:59.306575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5c6f0-95b2-4092-a505-c29ab25c8b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:59.309003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5c6f0-95b2-4092-a505-c29ab25c8b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:59.310236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5c6f0-95b2-4092-a505-c29ab25c8b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:59.315131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5c6f0-95b2-4092-a505-c29ab25c8b13 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:59.400185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 856/1704] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:21:59 2022] POST /identity/v3/projects => generated 363 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:59.404138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5c6f0-95b2-4092-a505-c29ab25c8b13 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:59.412684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:59.460841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b9288-1a17-4059-b813-f3ebee9fe393 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:59.561079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5c6f0-95b2-4092-a505-c29ab25c8b13 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:59.574916 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c4c2-69d5-4ef1-bcee-bb92fe6675ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:59.576960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c4c2-69d5-4ef1-bcee-bb92fe6675ff admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:59.577256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c4c2-69d5-4ef1-bcee-bb92fe6675ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:59.578546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c4c2-69d5-4ef1-bcee-bb92fe6675ff admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:59.580226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c4c2-69d5-4ef1-bcee-bb92fe6675ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:59.588538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c4c2-69d5-4ef1-bcee-bb92fe6675ff admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:59.602725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32217384-61ba-4f43-affc-d41a1a03b276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:59.604599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217384-61ba-4f43-affc-d41a1a03b276 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:59.605326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217384-61ba-4f43-affc-d41a1a03b276 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:59.606361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217384-61ba-4f43-affc-d41a1a03b276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:59.639317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32217384-61ba-4f43-affc-d41a1a03b276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:59.642186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32217384-61ba-4f43-affc-d41a1a03b276 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:59.646668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 857/1705] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:21:59 2022] POST /identity/v3/users => generated 288 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:21:59.647621 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c4c2-69d5-4ef1-bcee-bb92fe6675ff admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:59.681138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 849/1706] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:21:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:21:59.683648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32217384-61ba-4f43-affc-d41a1a03b276 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:59.685190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c4c2-69d5-4ef1-bcee-bb92fe6675ff admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:59.690587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-32217384-61ba-4f43-affc-d41a1a03b276 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:21:59.703118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e17f7-b8da-4f83-b164-11a44eb5dd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:59.704803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e17f7-b8da-4f83-b164-11a44eb5dd2a admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:59.705120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e17f7-b8da-4f83-b164-11a44eb5dd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:59.705395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e17f7-b8da-4f83-b164-11a44eb5dd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:59.706399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e17f7-b8da-4f83-b164-11a44eb5dd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:59.711798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e17f7-b8da-4f83-b164-11a44eb5dd2a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:59.737012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32217384-61ba-4f43-affc-d41a1a03b276 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:59.737282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 858/1707] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:21:59.739207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e17f7-b8da-4f83-b164-11a44eb5dd2a admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:59.756676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e17f7-b8da-4f83-b164-11a44eb5dd2a admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:21:59.759902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:59.760557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:59.760836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:59.761101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] PATH_INFO: `/v3/users/cf9a836a8ca64295a4289654378c2db6` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:59.762411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d53fb-7070-490f-a095-13394e172c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:21:59.763088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d53fb-7070-490f-a095-13394e172c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:21:59.763368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d53fb-7070-490f-a095-13394e172c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:21:59.763675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d53fb-7070-490f-a095-13394e172c00 admin admin] PATH_INFO: `/v3/projects/259508d5cb584fcda59e680e5222f1ff/users/a5315c3ac5a348d3b33065de79edf606/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:21:59.768582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9a836a8ca64295a4289654378c2db6)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:59.773364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:59.791836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d53fb-7070-490f-a095-13394e172c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=259508d5cb584fcda59e680e5222f1ff, user_id=a5315c3ac5a348d3b33065de79edf606, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:21:59.817153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d53fb-7070-490f-a095-13394e172c00 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:21:59.857976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 859/1708] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:21:59 2022] PUT /identity/v3/projects/259508d5cb584fcda59e680e5222f1ff/users/a5315c3ac5a348d3b33065de79edf606/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:21:59.859903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d53fb-7070-490f-a095-13394e172c00 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:21:59.943948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:21:59.957531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9a836a8ca64295a4289654378c2db6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:59.960380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9a836a8ca64295a4289654378c2db6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:21:59.982225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9a836a8ca64295a4289654378c2db6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:00.045133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9a836a8ca64295a4289654378c2db6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:00.046919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9a836a8ca64295a4289654378c2db6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:00.052142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d53fb-7070-490f-a095-13394e172c00 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:00.072953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ebf44-68cb-4276-b100-25bc59f02c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:00.074357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ebf44-68cb-4276-b100-25bc59f02c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:00.074788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ebf44-68cb-4276-b100-25bc59f02c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:00.075224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ebf44-68cb-4276-b100-25bc59f02c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:00.077205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ebf44-68cb-4276-b100-25bc59f02c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:00.079750 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ebf44-68cb-4276-b100-25bc59f02c20 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:00.119168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 850/1709] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:21:59 2022] DELETE /identity/v3/users/cf9a836a8ca64295a4289654378c2db6 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:00.119851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 860/1710] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:00.120613 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:00.122394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04037299-e78c-4d20-b29a-1993e7725e99 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:00.123897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04037299-e78c-4d20-b29a-1993e7725e99 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:00.124989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04037299-e78c-4d20-b29a-1993e7725e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:00.130010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:00.186403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-04037299-e78c-4d20-b29a-1993e7725e99 None None] MFA Rules not processed for user `1301a78c1a684c94acb9f26b56f39f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:00.190371 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-04037299-e78c-4d20-b29a-1993e7725e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:00.309236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8764e-3d4f-4494-a9f8-de59465090aa admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:00.318831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:00.327440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8a556-d0a2-4357-8e61-acca711d81d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:00.331003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8a556-d0a2-4357-8e61-acca711d81d3 service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:00.331297 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8a556-d0a2-4357-8e61-acca711d81d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:00.331573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8a556-d0a2-4357-8e61-acca711d81d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:00.335209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 861/1711] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:00.336309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04037299-e78c-4d20-b29a-1993e7725e99 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:00.340350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fca8a556-d0a2-4357-8e61-acca711d81d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:00.352548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-04037299-e78c-4d20-b29a-1993e7725e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:00.371725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8a556-d0a2-4357-8e61-acca711d81d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:00.373950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8a556-d0a2-4357-8e61-acca711d81d3 service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:00.398216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 851/1712] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:00.399635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cff77-0224-4893-b69f-92af4f1dd0d3 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:00.399922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cff77-0224-4893-b69f-92af4f1dd0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:00.400195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cff77-0224-4893-b69f-92af4f1dd0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:00.448082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-6f0cff77-0224-4893-b69f-92af4f1dd0d3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:00.449312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6f0cff77-0224-4893-b69f-92af4f1dd0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:00.596264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04037299-e78c-4d20-b29a-1993e7725e99 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:00.615063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0c78a-d55f-495b-b1f1-e778e97970b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:00.616357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0c78a-d55f-495b-b1f1-e778e97970b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:00.617243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0c78a-d55f-495b-b1f1-e778e97970b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:00.620281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0c78a-d55f-495b-b1f1-e778e97970b8 admin admin] PATH_INFO: `/v3/projects/259508d5cb584fcda59e680e5222f1ff/users/a5315c3ac5a348d3b33065de79edf606/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:00.632087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0c78a-d55f-495b-b1f1-e778e97970b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=259508d5cb584fcda59e680e5222f1ff, user_id=a5315c3ac5a348d3b33065de79edf606, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:00.636486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0c78a-d55f-495b-b1f1-e778e97970b8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:00.645959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 852/1713] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:00.681159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 862/1714] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:00 2022] PUT /identity/v3/projects/259508d5cb584fcda59e680e5222f1ff/users/a5315c3ac5a348d3b33065de79edf606/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:00.681354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cff77-0224-4893-b69f-92af4f1dd0d3 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:00.683278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c383c94-cdb2-41f3-8bc8-239971056151 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:00.683278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c383c94-cdb2-41f3-8bc8-239971056151 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:00.683471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c383c94-cdb2-41f3-8bc8-239971056151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:00.687324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6f0cff77-0224-4893-b69f-92af4f1dd0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:00.755168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-3c383c94-cdb2-41f3-8bc8-239971056151 None None] MFA Rules not processed for user `a5315c3ac5a348d3b33065de79edf606`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:00.756277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3c383c94-cdb2-41f3-8bc8-239971056151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:00.881063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cff77-0224-4893-b69f-92af4f1dd0d3 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:00.891415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:01.001506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 863/1715] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:01.019995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c383c94-cdb2-41f3-8bc8-239971056151 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:01.022607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3c383c94-cdb2-41f3-8bc8-239971056151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:01.034699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155b824-bfb6-4aa0-abe3-cc382a6f91aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:01.035461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155b824-bfb6-4aa0-abe3-cc382a6f91aa service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:01.035825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155b824-bfb6-4aa0-abe3-cc382a6f91aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:01.036188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155b824-bfb6-4aa0-abe3-cc382a6f91aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:01.223675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c383c94-cdb2-41f3-8bc8-239971056151 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:01.233339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:01.234624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:01.235101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:01.235895 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] PATH_INFO: `/v3/projects/fbc6ca1549a6487cbcbc8ef395cbc801` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:01.238968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc6ca1549a6487cbcbc8ef395cbc801)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:01.241419 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:01.306498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155b824-bfb6-4aa0-abe3-cc382a6f91aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:01.308316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155b824-bfb6-4aa0-abe3-cc382a6f91aa service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:01.313028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:01.338999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 853/1716] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:01.341412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155b824-bfb6-4aa0-abe3-cc382a6f91aa service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:01.353229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d155b824-bfb6-4aa0-abe3-cc382a6f91aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:01.357002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc6ca1549a6487cbcbc8ef395cbc801', 'request_id': 'req-6cf4064a-0a8c-4515-b2f4-612371f36fcd'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:01.386953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc6ca1549a6487cbcbc8ef395cbc801', 'request_id': 'req-6cf4064a-0a8c-4515-b2f4-612371f36fcd'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:01.398682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc6ca1549a6487cbcbc8ef395cbc801', 'request_id': 'req-6cf4064a-0a8c-4515-b2f4-612371f36fcd'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:01.442131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] The token cache is being invalidate because project fbc6ca1549a6487cbcbc8ef395cbc801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:01.442663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:01.449926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6cf4064a-0a8c-4515-b2f4-612371f36fcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:01.458757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 864/1717] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:01 2022] DELETE /identity/v3/projects/fbc6ca1549a6487cbcbc8ef395cbc801 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:01.458757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c68e3-5df2-4b81-8b95-da7db8add676 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:01.458757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c68e3-5df2-4b81-8b95-da7db8add676 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:01.458757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c68e3-5df2-4b81-8b95-da7db8add676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:01.580125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d85c68e3-5df2-4b81-8b95-da7db8add676 None None] MFA Rules not processed for user `a553cd4c2e154ee1979812ae575e6fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:01.581637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d85c68e3-5df2-4b81-8b95-da7db8add676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:01.624497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155b824-bfb6-4aa0-abe3-cc382a6f91aa service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:01.794047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 865/1718] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:01.798354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd6855-d01c-48ef-a338-18b146a655b8 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:01.802177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd6855-d01c-48ef-a338-18b146a655b8 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:01.802818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd6855-d01c-48ef-a338-18b146a655b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:01.826881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf937e6-96f1-4e38-a05b-39eec39334f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:01.830556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf937e6-96f1-4e38-a05b-39eec39334f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:01.831847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf937e6-96f1-4e38-a05b-39eec39334f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:01.832457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf937e6-96f1-4e38-a05b-39eec39334f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:01.844671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ecf937e6-96f1-4e38-a05b-39eec39334f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:01.877736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-12bd6855-d01c-48ef-a338-18b146a655b8 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:01.883981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-12bd6855-d01c-48ef-a338-18b146a655b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:01.968693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:02.103762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf937e6-96f1-4e38-a05b-39eec39334f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:02.110866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf937e6-96f1-4e38-a05b-39eec39334f9 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:02.141099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 866/1719] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:02.145660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83eede-4e04-4ecf-a568-9fd593ce2798 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:02.146078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83eede-4e04-4ecf-a568-9fd593ce2798 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:02.146320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83eede-4e04-4ecf-a568-9fd593ce2798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:02.176946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 854/1720] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 837 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:02.183715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f13217-1d1b-4660-91e1-12b487e7461d None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:02.185627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f13217-1d1b-4660-91e1-12b487e7461d None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:02.189548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f13217-1d1b-4660-91e1-12b487e7461d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:02.238125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7b83eede-4e04-4ecf-a568-9fd593ce2798 None None] MFA Rules not processed for user `a553cd4c2e154ee1979812ae575e6fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:02.245358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7b83eede-4e04-4ecf-a568-9fd593ce2798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:02.280588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-d2f13217-1d1b-4660-91e1-12b487e7461d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:02.291852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 867/1721] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:02.298294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d2f13217-1d1b-4660-91e1-12b487e7461d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:02.301666 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83eede-4e04-4ecf-a568-9fd593ce2798 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:02.310201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7b83eede-4e04-4ecf-a568-9fd593ce2798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:02.339103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 855/1722] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:02.343884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be1f8b-e7d3-4c85-9dac-192657fdb6c6 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:02.344164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be1f8b-e7d3-4c85-9dac-192657fdb6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:02.344447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be1f8b-e7d3-4c85-9dac-192657fdb6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:02.378113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83eede-4e04-4ecf-a568-9fd593ce2798 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:02.399096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:02.399626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:02.399901 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:02.400163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] PATH_INFO: `/v3/projects/6e970d9fb1b84d6299f48f1fb42d2b1d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:02.404895 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e970d9fb1b84d6299f48f1fb42d2b1d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:02.411590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:02.455374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f9be1f8b-e7d3-4c85-9dac-192657fdb6c6 None None] MFA Rules not processed for user `c28866ff5b2a4dd4ba05270b369f064e`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:02.458454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f9be1f8b-e7d3-4c85-9dac-192657fdb6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:02.496444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:02.551592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e970d9fb1b84d6299f48f1fb42d2b1d', 'request_id': 'req-02bfa9a6-83bf-4401-9873-8a75c481871f'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:02.580815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e970d9fb1b84d6299f48f1fb42d2b1d', 'request_id': 'req-02bfa9a6-83bf-4401-9873-8a75c481871f'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:02.599591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e970d9fb1b84d6299f48f1fb42d2b1d', 'request_id': 'req-02bfa9a6-83bf-4401-9873-8a75c481871f'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:02.656507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] The token cache is being invalidate because project 6e970d9fb1b84d6299f48f1fb42d2b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:02.656788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:02.670317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 856/1723] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:02.670632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:02.672588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be1f8b-e7d3-4c85-9dac-192657fdb6c6 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:02.674152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f9be1f8b-e7d3-4c85-9dac-192657fdb6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:02.682388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 868/1724] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:02 2022] DELETE /identity/v3/projects/6e970d9fb1b84d6299f48f1fb42d2b1d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:02.682388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:02.682388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:02.889109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfa9a6-83bf-4401-9873-8a75c481871f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:02.907968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:02.909965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:02.912187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:02.912624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:02.916465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:02.917161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:02.926437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:02.933493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be1f8b-e7d3-4c85-9dac-192657fdb6c6 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:03.002431 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 869/1725] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:22:03.004467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:03.024912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:03.071488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:03.144354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:03.145171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:03.145485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:03.146091 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:03.149444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:03.231601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e2b94-56a0-4f15-9d4a-7b88770fc371 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:03.242217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:03.242853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:03.243309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:03.243436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:03.246432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:03.264356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:03.266133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:03.289164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 857/1726] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:03.290181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:03.295593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:03.331211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870f322-29d5-477d-b8ec-c9a4ebe5a443 service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:03.340383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:03.341487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:03.341848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:03.342493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] PATH_INFO: `/v3/projects/74cb11b256ac497da3e8996bc8887351` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:03.349156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74cb11b256ac497da3e8996bc8887351)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:03.353425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:03.369933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:03.371481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:03.402808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 870/1727] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:03.403320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:03.412956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:03.454186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:03.470122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15285e68-cadf-4483-8492-8306fe1c2fa5 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:03.503881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74cb11b256ac497da3e8996bc8887351', 'request_id': 'req-9e9f6161-33e4-4d8f-ac4e-23791a782d36'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:03.521630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74cb11b256ac497da3e8996bc8887351', 'request_id': 'req-9e9f6161-33e4-4d8f-ac4e-23791a782d36'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:03.530137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74cb11b256ac497da3e8996bc8887351', 'request_id': 'req-9e9f6161-33e4-4d8f-ac4e-23791a782d36'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:03.551953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] The token cache is being invalidate because project 74cb11b256ac497da3e8996bc8887351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:03.552617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:03.554535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:03.561050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 858/1728] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:03 2022] DELETE /identity/v3/projects/74cb11b256ac497da3e8996bc8887351 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:03.562242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:03.565969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:03.706822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:03.729128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:03.729433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:03.729770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] PATH_INFO: `/v3/users/6fefeaae734b488ba769eec37d398a5d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:03.735046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fefeaae734b488ba769eec37d398a5d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:03.737186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:03.745955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f6161-33e4-4d8f-ac4e-23791a782d36 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:03.760902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154db1c-d085-4708-bb59-0814d1b2a59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:03.764698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154db1c-d085-4708-bb59-0814d1b2a59b admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:03.765792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154db1c-d085-4708-bb59-0814d1b2a59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:03.767000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154db1c-d085-4708-bb59-0814d1b2a59b admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:03.768216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154db1c-d085-4708-bb59-0814d1b2a59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:03.776962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154db1c-d085-4708-bb59-0814d1b2a59b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:03.802327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:03.838853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 859/1729] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:03 2022] POST /identity/v3/projects => generated 373 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:03.841234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154db1c-d085-4708-bb59-0814d1b2a59b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:03.846634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d154db1c-d085-4708-bb59-0814d1b2a59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:03.885193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fefeaae734b488ba769eec37d398a5d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:03.886957 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fefeaae734b488ba769eec37d398a5d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:03.922649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fefeaae734b488ba769eec37d398a5d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:03.923946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fefeaae734b488ba769eec37d398a5d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:03.953817 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fefeaae734b488ba769eec37d398a5d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:04.008566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 871/1730] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:03 2022] DELETE /identity/v3/users/6fefeaae734b488ba769eec37d398a5d => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:04.008566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:04.019484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:04.047170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154db1c-d085-4708-bb59-0814d1b2a59b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:04.150683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:04.151291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:04.151567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:04.151832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:04.154839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:04.240004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a71df3-c9bc-4d76-8e13-7e8b11932110 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:04.244878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:04.245544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:04.245831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:04.246105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] PATH_INFO: `/v3/users/e1ae8750ba654dee94fa60e4296c892c` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:04.252293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ae8750ba654dee94fa60e4296c892c)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:04.254351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:04.262316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:04.268181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:04.303729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 860/1731] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:04.304419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:04.310427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:04.315529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:04.391455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067aeac1-af8c-4bc1-8da6-704d0047e905 service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:04.395792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ae8750ba654dee94fa60e4296c892c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:04.397033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ae8750ba654dee94fa60e4296c892c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:04.411315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb50da2-d694-49e7-b87a-7eb13abdb8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:04.411991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb50da2-d694-49e7-b87a-7eb13abdb8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:04.412272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb50da2-d694-49e7-b87a-7eb13abdb8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:04.412990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb50da2-d694-49e7-b87a-7eb13abdb8b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:04.413181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb50da2-d694-49e7-b87a-7eb13abdb8b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:04.415175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb50da2-d694-49e7-b87a-7eb13abdb8b6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:04.433633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ae8750ba654dee94fa60e4296c892c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:04.434305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ae8750ba654dee94fa60e4296c892c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:04.455162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ae8750ba654dee94fa60e4296c892c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:04.482602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 861/1732] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:04 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:04.484991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17844fae-7add-4be8-9b66-cf25e2ab4f07 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:04.495390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17844fae-7add-4be8-9b66-cf25e2ab4f07 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:04.495390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17844fae-7add-4be8-9b66-cf25e2ab4f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:04.504289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 872/1733] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:04 2022] DELETE /identity/v3/users/e1ae8750ba654dee94fa60e4296c892c => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:04.504289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:04.507089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:04.530376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:04.670621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-17844fae-7add-4be8-9b66-cf25e2ab4f07 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:04.672123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-17844fae-7add-4be8-9b66-cf25e2ab4f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:04.738345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1255f0-7da1-436d-9975-a06f09b3a8a0 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:04.772640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ebde2-7323-44ca-adda-042ef1f49d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:04.772640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ebde2-7323-44ca-adda-042ef1f49d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:04.772640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ebde2-7323-44ca-adda-042ef1f49d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:04.772640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ebde2-7323-44ca-adda-042ef1f49d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:04.773049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ebde2-7323-44ca-adda-042ef1f49d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:04.775778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ebde2-7323-44ca-adda-042ef1f49d7b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:04.821605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 873/1734] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:04.822911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ebde2-7323-44ca-adda-042ef1f49d7b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:04.858997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ebde2-7323-44ca-adda-042ef1f49d7b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:04.869259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb23da-b4ec-4b97-8a99-804f3791dd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:04.870077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb23da-b4ec-4b97-8a99-804f3791dd7a admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:04.870506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb23da-b4ec-4b97-8a99-804f3791dd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:04.870878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb23da-b4ec-4b97-8a99-804f3791dd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:04.871563 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb23da-b4ec-4b97-8a99-804f3791dd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:04.889589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb23da-b4ec-4b97-8a99-804f3791dd7a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:04.955611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 874/1735] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:04.960452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb23da-b4ec-4b97-8a99-804f3791dd7a admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:04.982191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb23da-b4ec-4b97-8a99-804f3791dd7a admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:04.988011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 862/1736] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:22:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:05.023633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f86a2-dc96-4894-a978-255c2d9b8725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:05.025848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f86a2-dc96-4894-a978-255c2d9b8725 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:05.028739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f86a2-dc96-4894-a978-255c2d9b8725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:05.030147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f86a2-dc96-4894-a978-255c2d9b8725 admin admin] PATH_INFO: `/v3/projects/032eacedb6de402cb9cca2453f68e9a7/users/fa0c6e57932b4f3fa952a69e3cb9203a/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:05.057273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0d0ac-8ce5-4acd-b17c-2f0c08c768b7 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:05.057573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0d0ac-8ce5-4acd-b17c-2f0c08c768b7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:05.057849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0d0ac-8ce5-4acd-b17c-2f0c08c768b7 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:05.059205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 863/1737] 10.222.0.23 () {54 vars in 912 bytes} [Wed Oct 5 20:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:22:05.065477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659b67-fb3b-4d45-8dab-b85b418e6297 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:05.065781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659b67-fb3b-4d45-8dab-b85b418e6297 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:05.066058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659b67-fb3b-4d45-8dab-b85b418e6297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:05.103567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-70659b67-fb3b-4d45-8dab-b85b418e6297 None None] MFA Rules not processed for user `8f5a88f263014f15a6f59cfdaa2bb6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:05.104669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f86a2-dc96-4894-a978-255c2d9b8725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032eacedb6de402cb9cca2453f68e9a7, user_id=fa0c6e57932b4f3fa952a69e3cb9203a, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:05.106286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-70659b67-fb3b-4d45-8dab-b85b418e6297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:05.108249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f86a2-dc96-4894-a978-255c2d9b8725 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:05.134505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 875/1738] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:04 2022] PUT /identity/v3/projects/032eacedb6de402cb9cca2453f68e9a7/users/fa0c6e57932b4f3fa952a69e3cb9203a/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:05.147849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb0fbc-c07b-4d2b-9723-432a1304f670 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:05.147849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb0fbc-c07b-4d2b-9723-432a1304f670 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:05.148380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb0fbc-c07b-4d2b-9723-432a1304f670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:05.228431 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-34bb0fbc-c07b-4d2b-9723-432a1304f670 None None] MFA Rules not processed for user `fa0c6e57932b4f3fa952a69e3cb9203a`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:05.230713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-34bb0fbc-c07b-4d2b-9723-432a1304f670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:05.244931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 864/1739] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:22:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:05.342447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659b67-fb3b-4d45-8dab-b85b418e6297 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:05.344229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-70659b67-fb3b-4d45-8dab-b85b418e6297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:05.461072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:05.500898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 876/1740] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:05.503075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb0fbc-c07b-4d2b-9723-432a1304f670 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:05.506534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-34bb0fbc-c07b-4d2b-9723-432a1304f670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:05.686905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659b67-fb3b-4d45-8dab-b85b418e6297 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:05.696647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb0fbc-c07b-4d2b-9723-432a1304f670 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:05.705784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d27ab8-8be5-4184-883f-bc9df9898e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:05.706341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27ab8-8be5-4184-883f-bc9df9898e2f service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:05.706615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27ab8-8be5-4184-883f-bc9df9898e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:05.706886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27ab8-8be5-4184-883f-bc9df9898e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:05.709240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:05.754091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d27ab8-8be5-4184-883f-bc9df9898e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:05.755990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d27ab8-8be5-4184-883f-bc9df9898e2f service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:05.787435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 877/1741] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:05.788281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d27ab8-8be5-4184-883f-bc9df9898e2f service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:05.938687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3c20-243d-447a-9df7-8d4980b1ab4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:05.939328 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3c20-243d-447a-9df7-8d4980b1ab4a service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:05.939860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3c20-243d-447a-9df7-8d4980b1ab4a service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:05.940136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3c20-243d-447a-9df7-8d4980b1ab4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:05.978733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d27ab8-8be5-4184-883f-bc9df9898e2f service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:05.995549 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357e91c-da0a-4685-b251-c4816493537d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:05.996187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357e91c-da0a-4685-b251-c4816493537d admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:05.996467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357e91c-da0a-4685-b251-c4816493537d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:05.996741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357e91c-da0a-4685-b251-c4816493537d admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:05.997585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357e91c-da0a-4685-b251-c4816493537d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:06.003180 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357e91c-da0a-4685-b251-c4816493537d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:06.113167 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 878/1742] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:05 2022] POST /identity/v3/projects => generated 373 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:06.117101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357e91c-da0a-4685-b251-c4816493537d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:06.121946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c357e91c-da0a-4685-b251-c4816493537d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:06.177224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f3c20-243d-447a-9df7-8d4980b1ab4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:06.181760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f3c20-243d-447a-9df7-8d4980b1ab4a service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:06.200887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 865/1743] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:22:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 860 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:22:06.201791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3c20-243d-447a-9df7-8d4980b1ab4a service swift] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:06.227173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3c20-243d-447a-9df7-8d4980b1ab4a service swift] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:06.374044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357e91c-da0a-4685-b251-c4816493537d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:06.395974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:06.399812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:06.400121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:06.400577 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] PATH_INFO: `/v3/projects/316bdaecc03c4a7d9e9dc91dd32ca85a` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:06.403492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=316bdaecc03c4a7d9e9dc91dd32ca85a)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:06.405686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:06.444127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029313e-b02e-4e55-b93b-e5d04b361e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:06.444936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029313e-b02e-4e55-b93b-e5d04b361e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:06.445222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029313e-b02e-4e55-b93b-e5d04b361e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:06.445500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029313e-b02e-4e55-b93b-e5d04b361e7f admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:06.446227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029313e-b02e-4e55-b93b-e5d04b361e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:06.452339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029313e-b02e-4e55-b93b-e5d04b361e7f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:06.491540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:06.513277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 866/1744] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:06 2022] POST /identity/v3/users => generated 293 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:06.518701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029313e-b02e-4e55-b93b-e5d04b361e7f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:06.552433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029313e-b02e-4e55-b93b-e5d04b361e7f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:06.587945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316bdaecc03c4a7d9e9dc91dd32ca85a', 'request_id': 'req-64434451-1ce5-480f-a0b0-cfb4eb2dc058'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:06.618457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316bdaecc03c4a7d9e9dc91dd32ca85a', 'request_id': 'req-64434451-1ce5-480f-a0b0-cfb4eb2dc058'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:06.640918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316bdaecc03c4a7d9e9dc91dd32ca85a', 'request_id': 'req-64434451-1ce5-480f-a0b0-cfb4eb2dc058'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:06.742172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] The token cache is being invalidate because project 316bdaecc03c4a7d9e9dc91dd32ca85a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:06.742527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:06.743681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64434451-1ce5-480f-a0b0-cfb4eb2dc058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:06.751478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 879/1745] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:06 2022] DELETE /identity/v3/projects/316bdaecc03c4a7d9e9dc91dd32ca85a => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:06.757086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1dc738-6b67-456b-ba21-5804b21b7575 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:06.757336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1dc738-6b67-456b-ba21-5804b21b7575 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:06.757709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1dc738-6b67-456b-ba21-5804b21b7575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:06.768113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:06.797693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03285e7d-39df-4cf8-bd44-7e246c787d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:06.798573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03285e7d-39df-4cf8-bd44-7e246c787d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:06.798972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03285e7d-39df-4cf8-bd44-7e246c787d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:06.799265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03285e7d-39df-4cf8-bd44-7e246c787d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:06.800132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03285e7d-39df-4cf8-bd44-7e246c787d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:06.803191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03285e7d-39df-4cf8-bd44-7e246c787d6f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:06.820778 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 867/1746] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:06.829880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03285e7d-39df-4cf8-bd44-7e246c787d6f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:06.833969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-03285e7d-39df-4cf8-bd44-7e246c787d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:06.861444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-4e1dc738-6b67-456b-ba21-5804b21b7575 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:06.875506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03285e7d-39df-4cf8-bd44-7e246c787d6f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:06.877790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4e1dc738-6b67-456b-ba21-5804b21b7575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:06.893720 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0392f-7db3-4c73-811f-fb21273a64e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:06.897687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0392f-7db3-4c73-811f-fb21273a64e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:06.899018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0392f-7db3-4c73-811f-fb21273a64e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:06.900254 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0392f-7db3-4c73-811f-fb21273a64e4 admin admin] PATH_INFO: `/v3/projects/8cf48a5a7e2c4ffd85d97ffc5c581613/users/cd05c7d44369424080a25977e0a92019/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:06.909474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 880/1747] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:06.920545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789c6cc-c7e4-4a4c-8de3-494d40560064 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:06.923769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789c6cc-c7e4-4a4c-8de3-494d40560064 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:06.924123 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789c6cc-c7e4-4a4c-8de3-494d40560064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:06.971759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0392f-7db3-4c73-811f-fb21273a64e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf48a5a7e2c4ffd85d97ffc5c581613, user_id=cd05c7d44369424080a25977e0a92019, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:06.974987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0392f-7db3-4c73-811f-fb21273a64e4 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:06.998284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a789c6cc-c7e4-4a4c-8de3-494d40560064 None None] MFA Rules not processed for user `ff9b14ab630d4a928614c60e6aea667c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:06.999417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a789c6cc-c7e4-4a4c-8de3-494d40560064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:07.020458 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 868/1748] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:06 2022] PUT /identity/v3/projects/8cf48a5a7e2c4ffd85d97ffc5c581613/users/cd05c7d44369424080a25977e0a92019/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:07.022282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496eff8-da7f-44d2-9fe3-a9f024747c63 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:07.022572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496eff8-da7f-44d2-9fe3-a9f024747c63 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:07.022849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496eff8-da7f-44d2-9fe3-a9f024747c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:07.090854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4496eff8-da7f-44d2-9fe3-a9f024747c63 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:07.092902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4496eff8-da7f-44d2-9fe3-a9f024747c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:07.245945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 881/1749] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:07.256295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789c6cc-c7e4-4a4c-8de3-494d40560064 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:07.305993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789c6cc-c7e4-4a4c-8de3-494d40560064 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:07.444700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 869/1750] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:07.446781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496eff8-da7f-44d2-9fe3-a9f024747c63 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:07.479161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496eff8-da7f-44d2-9fe3-a9f024747c63 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:07.497474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b98c7b-569b-4be6-9bea-de3084e192ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:07.498886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b98c7b-569b-4be6-9bea-de3084e192ca admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:07.500261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b98c7b-569b-4be6-9bea-de3084e192ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:07.500656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b98c7b-569b-4be6-9bea-de3084e192ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:07.501421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b98c7b-569b-4be6-9bea-de3084e192ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:07.504820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b98c7b-569b-4be6-9bea-de3084e192ca admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:07.510670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db077c04-1236-4abc-8250-012e70cd96b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:07.511360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db077c04-1236-4abc-8250-012e70cd96b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:07.511664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db077c04-1236-4abc-8250-012e70cd96b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:07.511946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db077c04-1236-4abc-8250-012e70cd96b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:07.512602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db077c04-1236-4abc-8250-012e70cd96b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:07.512901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db077c04-1236-4abc-8250-012e70cd96b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:07.524439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db077c04-1236-4abc-8250-012e70cd96b3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:07.553084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 882/1751] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:07.557597 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b98c7b-569b-4be6-9bea-de3084e192ca admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:07.581489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 870/1752] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:22:07.585886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db077c04-1236-4abc-8250-012e70cd96b3 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:07.588337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b98c7b-569b-4be6-9bea-de3084e192ca admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:07.609922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65599c35-a816-42d8-86b9-54c971bd1a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:07.610762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65599c35-a816-42d8-86b9-54c971bd1a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:07.611160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65599c35-a816-42d8-86b9-54c971bd1a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:07.611534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65599c35-a816-42d8-86b9-54c971bd1a42 admin admin] PATH_INFO: `/v3/projects/8cf48a5a7e2c4ffd85d97ffc5c581613/users/cd05c7d44369424080a25977e0a92019/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:07.617899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65599c35-a816-42d8-86b9-54c971bd1a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf48a5a7e2c4ffd85d97ffc5c581613, user_id=cd05c7d44369424080a25977e0a92019, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:07.639912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65599c35-a816-42d8-86b9-54c971bd1a42 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:07.643762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db077c04-1236-4abc-8250-012e70cd96b3 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:07.652524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab96bb7-e5f2-429b-a14c-97cda7296605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:07.653611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab96bb7-e5f2-429b-a14c-97cda7296605 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:07.654105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab96bb7-e5f2-429b-a14c-97cda7296605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:07.654658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab96bb7-e5f2-429b-a14c-97cda7296605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:07.655509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab96bb7-e5f2-429b-a14c-97cda7296605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:07.662091 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab96bb7-e5f2-429b-a14c-97cda7296605 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:07.690437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 883/1753] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:07 2022] PUT /identity/v3/projects/8cf48a5a7e2c4ffd85d97ffc5c581613/users/cd05c7d44369424080a25977e0a92019/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:07.706213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024386ec-96bc-4859-b1af-588ce10fc763 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:07.706213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024386ec-96bc-4859-b1af-588ce10fc763 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:07.706213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024386ec-96bc-4859-b1af-588ce10fc763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:07.780620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 871/1754] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:07 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:07.787578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab96bb7-e5f2-429b-a14c-97cda7296605 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:07.796352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-024386ec-96bc-4859-b1af-588ce10fc763 None None] MFA Rules not processed for user `cd05c7d44369424080a25977e0a92019`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:07.803554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-024386ec-96bc-4859-b1af-588ce10fc763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:07.850703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:07.976889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 884/1755] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:07.990607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024386ec-96bc-4859-b1af-588ce10fc763 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:07.994103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab96bb7-e5f2-429b-a14c-97cda7296605 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:07.994478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-024386ec-96bc-4859-b1af-588ce10fc763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:08.012831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944b75e-75e5-4095-b2c7-7c837ad36999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:08.014009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944b75e-75e5-4095-b2c7-7c837ad36999 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:08.014456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944b75e-75e5-4095-b2c7-7c837ad36999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:08.015078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944b75e-75e5-4095-b2c7-7c837ad36999 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:08.016052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944b75e-75e5-4095-b2c7-7c837ad36999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:08.024364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944b75e-75e5-4095-b2c7-7c837ad36999 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:08.107190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 872/1756] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:07 2022] POST /identity/v3/users => generated 294 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:08.107190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944b75e-75e5-4095-b2c7-7c837ad36999 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:08.135954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944b75e-75e5-4095-b2c7-7c837ad36999 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:08.174439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1eca0-cded-40d6-a0ce-d6ab0e474624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:08.175078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1eca0-cded-40d6-a0ce-d6ab0e474624 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:08.175353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1eca0-cded-40d6-a0ce-d6ab0e474624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:08.175621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1eca0-cded-40d6-a0ce-d6ab0e474624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:08.176463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1eca0-cded-40d6-a0ce-d6ab0e474624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:08.189841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1eca0-cded-40d6-a0ce-d6ab0e474624 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:08.199456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:08.211024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024386ec-96bc-4859-b1af-588ce10fc763 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:08.221635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c01a6-f220-458c-a870-ed3fc761c9d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:08.222269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c01a6-f220-458c-a870-ed3fc761c9d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:08.222548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c01a6-f220-458c-a870-ed3fc761c9d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:08.222923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c01a6-f220-458c-a870-ed3fc761c9d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:08.246438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c01a6-f220-458c-a870-ed3fc761c9d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:08.247960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c01a6-f220-458c-a870-ed3fc761c9d0 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:08.254353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 873/1757] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:08.261298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1eca0-cded-40d6-a0ce-d6ab0e474624 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:08.289450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 885/1758] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:08.303768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1eca0-cded-40d6-a0ce-d6ab0e474624 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:08.329440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993dca0-5609-4188-aea3-3dae607ee150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:08.339073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993dca0-5609-4188-aea3-3dae607ee150 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:08.339358 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993dca0-5609-4188-aea3-3dae607ee150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:08.339663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993dca0-5609-4188-aea3-3dae607ee150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:08.340377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993dca0-5609-4188-aea3-3dae607ee150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:08.358130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0993dca0-5609-4188-aea3-3dae607ee150 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:08.416173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 874/1759] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:08.417187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c01a6-f220-458c-a870-ed3fc761c9d0 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:08.433808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c01a6-f220-458c-a870-ed3fc761c9d0 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:08.439565 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993dca0-5609-4188-aea3-3dae607ee150 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:08.455598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79caa49e-4210-4307-b6e8-8dc50f4a9958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:08.458027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993dca0-5609-4188-aea3-3dae607ee150 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:08.458667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caa49e-4210-4307-b6e8-8dc50f4a9958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:08.459864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caa49e-4210-4307-b6e8-8dc50f4a9958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:08.460146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caa49e-4210-4307-b6e8-8dc50f4a9958 admin admin] PATH_INFO: `/v3/projects/c977c2d5d91445c2a431e8384a285387/users/66154798e691421aa922386969ac6f65/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:08.475349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d218dca-c0ac-4484-a961-ae39c9f47d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:08.476162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218dca-c0ac-4484-a961-ae39c9f47d39 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:08.476491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218dca-c0ac-4484-a961-ae39c9f47d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:08.476770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d218dca-c0ac-4484-a961-ae39c9f47d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:08.484761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3d218dca-c0ac-4484-a961-ae39c9f47d39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:08.539322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79caa49e-4210-4307-b6e8-8dc50f4a9958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c977c2d5d91445c2a431e8384a285387, user_id=66154798e691421aa922386969ac6f65, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:08.542067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79caa49e-4210-4307-b6e8-8dc50f4a9958 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:08.594333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 886/1760] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:08 2022] PUT /identity/v3/projects/c977c2d5d91445c2a431e8384a285387/users/66154798e691421aa922386969ac6f65/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:08.604620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16975a2-5a83-46d2-85c9-9046d4e72dc3 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:08.605472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16975a2-5a83-46d2-85c9-9046d4e72dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:08.606142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16975a2-5a83-46d2-85c9-9046d4e72dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:08.640593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218dca-c0ac-4484-a961-ae39c9f47d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:08.643958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d218dca-c0ac-4484-a961-ae39c9f47d39 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:08.670265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 875/1761] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:08.707347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a16975a2-5a83-46d2-85c9-9046d4e72dc3 None None] MFA Rules not processed for user `66154798e691421aa922386969ac6f65`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:08.708855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a16975a2-5a83-46d2-85c9-9046d4e72dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:08.963292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 887/1762] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:08.965440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d218dca-c0ac-4484-a961-ae39c9f47d39 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:08.988910 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d218dca-c0ac-4484-a961-ae39c9f47d39 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:09.123404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16975a2-5a83-46d2-85c9-9046d4e72dc3 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:09.125032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a16975a2-5a83-46d2-85c9-9046d4e72dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:09.164388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a201cd0-b397-4de2-bdc6-83d0f12757a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:09.165219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a201cd0-b397-4de2-bdc6-83d0f12757a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:09.165584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a201cd0-b397-4de2-bdc6-83d0f12757a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:09.168074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a201cd0-b397-4de2-bdc6-83d0f12757a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:09.169254 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a201cd0-b397-4de2-bdc6-83d0f12757a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:09.176226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a201cd0-b397-4de2-bdc6-83d0f12757a3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:09.213168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:09.250790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 876/1763] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:08 2022] POST /identity/v3/projects => generated 387 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:09.254547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a201cd0-b397-4de2-bdc6-83d0f12757a3 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:09.377786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16975a2-5a83-46d2-85c9-9046d4e72dc3 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:09.497207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a201cd0-b397-4de2-bdc6-83d0f12757a3 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:09.513406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a7b8d-15a2-4b11-b1e8-7b408832e99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:09.514263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a7b8d-15a2-4b11-b1e8-7b408832e99a admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:09.514882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a7b8d-15a2-4b11-b1e8-7b408832e99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:09.514882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a7b8d-15a2-4b11-b1e8-7b408832e99a admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:09.515699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a7b8d-15a2-4b11-b1e8-7b408832e99a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:09.518906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a7b8d-15a2-4b11-b1e8-7b408832e99a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:09.542373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:09.552606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 877/1764] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:09 2022] POST /identity/v3/users => generated 300 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:09.554451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:09.555769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:09.556188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:09.556724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] PATH_INFO: `/v3/projects/17fd8aeb882347af98589aef1b293e6a` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:09.556837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a7b8d-15a2-4b11-b1e8-7b408832e99a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:09.558578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17fd8aeb882347af98589aef1b293e6a)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:09.562202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:09.603516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a7b8d-15a2-4b11-b1e8-7b408832e99a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:09.624783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c50b9-88ae-424b-8b16-997cf1daa753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:09.626481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c50b9-88ae-424b-8b16-997cf1daa753 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:09.627128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c50b9-88ae-424b-8b16-997cf1daa753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:09.627625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c50b9-88ae-424b-8b16-997cf1daa753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:09.629839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c50b9-88ae-424b-8b16-997cf1daa753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:09.634872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c50b9-88ae-424b-8b16-997cf1daa753 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:09.665998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 878/1765] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:09.667903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17fd8aeb882347af98589aef1b293e6a', 'request_id': 'req-02185c18-d179-47cb-83ac-1b552f211e33'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:09.671051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c50b9-88ae-424b-8b16-997cf1daa753 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:09.693144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17fd8aeb882347af98589aef1b293e6a', 'request_id': 'req-02185c18-d179-47cb-83ac-1b552f211e33'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:09.704156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c50b9-88ae-424b-8b16-997cf1daa753 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:09.706189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17fd8aeb882347af98589aef1b293e6a', 'request_id': 'req-02185c18-d179-47cb-83ac-1b552f211e33'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:09.746834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] The token cache is being invalidate because project 17fd8aeb882347af98589aef1b293e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:09.747310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:09.749344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-02185c18-d179-47cb-83ac-1b552f211e33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:09.754323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 888/1766] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:09 2022] DELETE /identity/v3/projects/17fd8aeb882347af98589aef1b293e6a => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:09.756722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b180fb-a0fb-4d16-a93f-d4732132a240 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:09.757140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b180fb-a0fb-4d16-a93f-d4732132a240 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:09.757683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b180fb-a0fb-4d16-a93f-d4732132a240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:09.818504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-53b180fb-a0fb-4d16-a93f-d4732132a240 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:09.825453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-53b180fb-a0fb-4d16-a93f-d4732132a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:09.861230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce916bee-b95f-4850-b13a-66c0ee6d6bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:09.864491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce916bee-b95f-4850-b13a-66c0ee6d6bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:09.871076 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce916bee-b95f-4850-b13a-66c0ee6d6bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:09.871447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce916bee-b95f-4850-b13a-66c0ee6d6bbd admin admin] PATH_INFO: `/v3/projects/d39bb953cefb4cee865efe783de6d2f9/users/d65c6109416f4ceca10ec0ebdb30cf71/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:09.924337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce916bee-b95f-4850-b13a-66c0ee6d6bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39bb953cefb4cee865efe783de6d2f9, user_id=d65c6109416f4ceca10ec0ebdb30cf71, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:09.927449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce916bee-b95f-4850-b13a-66c0ee6d6bbd admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:09.968519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 879/1767] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:09 2022] PUT /identity/v3/projects/d39bb953cefb4cee865efe783de6d2f9/users/d65c6109416f4ceca10ec0ebdb30cf71/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:09.969702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce916bee-b95f-4850-b13a-66c0ee6d6bbd admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:09.972631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ce916bee-b95f-4850-b13a-66c0ee6d6bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:10.014067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 889/1768] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:10.016633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd5119-8cf4-4765-be0b-b9a9858bda2f None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.016913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd5119-8cf4-4765-be0b-b9a9858bda2f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.017187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd5119-8cf4-4765-be0b-b9a9858bda2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.041943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:10.071511 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e7bd5119-8cf4-4765-be0b-b9a9858bda2f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:10.072758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e7bd5119-8cf4-4765-be0b-b9a9858bda2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:10.096213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 890/1769] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:10.097374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd5119-8cf4-4765-be0b-b9a9858bda2f None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:10.097783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce916bee-b95f-4850-b13a-66c0ee6d6bbd admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.115349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd5119-8cf4-4765-be0b-b9a9858bda2f None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.124255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90ee4e-7102-4965-9cd7-1f1d853d139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.126206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90ee4e-7102-4965-9cd7-1f1d853d139e admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.126661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90ee4e-7102-4965-9cd7-1f1d853d139e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.127022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90ee4e-7102-4965-9cd7-1f1d853d139e admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.127781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90ee4e-7102-4965-9cd7-1f1d853d139e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.128285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a5af0-1817-42ed-9a37-dbd1a52dbceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.128947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5af0-1817-42ed-9a37-dbd1a52dbceb admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.129224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5af0-1817-42ed-9a37-dbd1a52dbceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.129536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5af0-1817-42ed-9a37-dbd1a52dbceb admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.130328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a5af0-1817-42ed-9a37-dbd1a52dbceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:10.130450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90ee4e-7102-4965-9cd7-1f1d853d139e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.130674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a5af0-1817-42ed-9a37-dbd1a52dbceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.132270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a5af0-1817-42ed-9a37-dbd1a52dbceb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.166401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 891/1770] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:22:10.166614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 880/1771] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:10.168399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a5af0-1817-42ed-9a37-dbd1a52dbceb admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:10.174019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90ee4e-7102-4965-9cd7-1f1d853d139e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:10.184711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a5af0-1817-42ed-9a37-dbd1a52dbceb admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.190356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bab7d-e9fe-46de-8ade-153941ec13f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.191629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bab7d-e9fe-46de-8ade-153941ec13f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.192374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bab7d-e9fe-46de-8ade-153941ec13f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.192853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bab7d-e9fe-46de-8ade-153941ec13f0 admin admin] PATH_INFO: `/v3/projects/d39bb953cefb4cee865efe783de6d2f9/users/d65c6109416f4ceca10ec0ebdb30cf71/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.195353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bab7d-e9fe-46de-8ade-153941ec13f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39bb953cefb4cee865efe783de6d2f9, user_id=d65c6109416f4ceca10ec0ebdb30cf71, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.199744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90ee4e-7102-4965-9cd7-1f1d853d139e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.203068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bab7d-e9fe-46de-8ade-153941ec13f0 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.215614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cda97-1b70-4dd2-87c5-9fe586d210c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.216355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cda97-1b70-4dd2-87c5-9fe586d210c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.217207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cda97-1b70-4dd2-87c5-9fe586d210c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.217482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cda97-1b70-4dd2-87c5-9fe586d210c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.218144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cda97-1b70-4dd2-87c5-9fe586d210c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.221432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cda97-1b70-4dd2-87c5-9fe586d210c3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.249811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 892/1772] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:10 2022] PUT /identity/v3/projects/d39bb953cefb4cee865efe783de6d2f9/users/d65c6109416f4ceca10ec0ebdb30cf71/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:10.252353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d115e0-b5e3-4807-b62c-566f0f0fb90e None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.252592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d115e0-b5e3-4807-b62c-566f0f0fb90e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.252937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d115e0-b5e3-4807-b62c-566f0f0fb90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.296905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 881/1773] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:10 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:10.298752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0cc04-0476-4f6e-999c-282d7e650a92 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.299088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0cc04-0476-4f6e-999c-282d7e650a92 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.299427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0cc04-0476-4f6e-999c-282d7e650a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.314366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-87d115e0-b5e3-4807-b62c-566f0f0fb90e None None] MFA Rules not processed for user `c28866ff5b2a4dd4ba05270b369f064e`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:10.315743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-87d115e0-b5e3-4807-b62c-566f0f0fb90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:10.352626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-fea0cc04-0476-4f6e-999c-282d7e650a92 None None] MFA Rules not processed for user `d65c6109416f4ceca10ec0ebdb30cf71`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:10.363889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fea0cc04-0476-4f6e-999c-282d7e650a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:10.433784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 893/1774] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:10.434876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d115e0-b5e3-4807-b62c-566f0f0fb90e None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:10.441421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:10.591915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 882/1775] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:10.626743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0cc04-0476-4f6e-999c-282d7e650a92 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:10.628292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fea0cc04-0476-4f6e-999c-282d7e650a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:10.632991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d115e0-b5e3-4807-b62c-566f0f0fb90e None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.640303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbaa548-45d0-4515-83cf-4471ae794186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.640938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaa548-45d0-4515-83cf-4471ae794186 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.641216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaa548-45d0-4515-83cf-4471ae794186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.641489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbaa548-45d0-4515-83cf-4471ae794186 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.648273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbaa548-45d0-4515-83cf-4471ae794186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.650294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbaa548-45d0-4515-83cf-4471ae794186 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.718086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 894/1776] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:10 2022] POST /identity/v3/users => generated 293 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:10.722587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbaa548-45d0-4515-83cf-4471ae794186 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:10.748277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbaa548-45d0-4515-83cf-4471ae794186 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.760314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f5fbc-dd96-4f15-844d-954656ab5c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.760942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5fbc-dd96-4f15-844d-954656ab5c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.761222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5fbc-dd96-4f15-844d-954656ab5c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.761507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5fbc-dd96-4f15-844d-954656ab5c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.762172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f5fbc-dd96-4f15-844d-954656ab5c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.770082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f5fbc-dd96-4f15-844d-954656ab5c51 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.771115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0cc04-0476-4f6e-999c-282d7e650a92 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.793387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dff1c8-21a6-4e34-b160-1f5908691b58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.797141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dff1c8-21a6-4e34-b160-1f5908691b58 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.799689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dff1c8-21a6-4e34-b160-1f5908691b58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.800153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dff1c8-21a6-4e34-b160-1f5908691b58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.811914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 895/1777] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:10.814999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f5fbc-dd96-4f15-844d-954656ab5c51 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:10.827366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dff1c8-21a6-4e34-b160-1f5908691b58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.829084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dff1c8-21a6-4e34-b160-1f5908691b58 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.833457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f5fbc-dd96-4f15-844d-954656ab5c51 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.839878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd94129-c3a4-4958-ab01-b4bc8b4d7eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.840492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd94129-c3a4-4958-ab01-b4bc8b4d7eaa admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.840763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd94129-c3a4-4958-ab01-b4bc8b4d7eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.848449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 883/1778] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:10.852535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd94129-c3a4-4958-ab01-b4bc8b4d7eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.853617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd94129-c3a4-4958-ab01-b4bc8b4d7eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.858086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd94129-c3a4-4958-ab01-b4bc8b4d7eaa admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.873429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 896/1779] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:10.877178 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dff1c8-21a6-4e34-b160-1f5908691b58 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:10.900540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dff1c8-21a6-4e34-b160-1f5908691b58 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.908492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e327c1d-a28f-4a04-a310-7e011054446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.909228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327c1d-a28f-4a04-a310-7e011054446f admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.909599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327c1d-a28f-4a04-a310-7e011054446f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.910056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327c1d-a28f-4a04-a310-7e011054446f admin admin] PATH_INFO: `/v3/projects/288266d288794c67a86a1c3377e5cafa/users/b77d9421d0fd4eb899fc8bb133ff10ff/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.933994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd94129-c3a4-4958-ab01-b4bc8b4d7eaa admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:10.943524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e327c1d-a28f-4a04-a310-7e011054446f admin admin] RBAC: Authorizing `identity:create_grant(project_id=288266d288794c67a86a1c3377e5cafa, user_id=b77d9421d0fd4eb899fc8bb133ff10ff, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.946264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e327c1d-a28f-4a04-a310-7e011054446f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.960119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd94129-c3a4-4958-ab01-b4bc8b4d7eaa admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:10.971832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:10.973350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.973888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.974237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] PATH_INFO: `/v3/users/66154798e691421aa922386969ac6f65` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:10.979451 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 884/1780] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:10 2022] PUT /identity/v3/projects/288266d288794c67a86a1c3377e5cafa/users/b77d9421d0fd4eb899fc8bb133ff10ff/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:10.980438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66154798e691421aa922386969ac6f65)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:10.984355 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:10.991869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef3f71-fec5-4a11-a43c-e61ab45a688b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:10.992725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef3f71-fec5-4a11-a43c-e61ab45a688b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:10.995010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef3f71-fec5-4a11-a43c-e61ab45a688b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:11.053351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-67ef3f71-fec5-4a11-a43c-e61ab45a688b None None] MFA Rules not processed for user `b77d9421d0fd4eb899fc8bb133ff10ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:11.054625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-67ef3f71-fec5-4a11-a43c-e61ab45a688b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:11.084542 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:11.110884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66154798e691421aa922386969ac6f65'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:11.113072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66154798e691421aa922386969ac6f65'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:11.129541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66154798e691421aa922386969ac6f65'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:11.132469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66154798e691421aa922386969ac6f65'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:11.152612 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f6309e64-f372-483a-a335-8f49597a3976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66154798e691421aa922386969ac6f65'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:11.193756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 897/1781] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:10 2022] DELETE /identity/v3/users/66154798e691421aa922386969ac6f65 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:11.198879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c130c5-8bbc-44c3-955d-06ee46603331 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:11.199166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c130c5-8bbc-44c3-955d-06ee46603331 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:11.199443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c130c5-8bbc-44c3-955d-06ee46603331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:11.228011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 885/1782] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:11.234012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-70c130c5-8bbc-44c3-955d-06ee46603331 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:11.235508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-70c130c5-8bbc-44c3-955d-06ee46603331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:11.251498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef3f71-fec5-4a11-a43c-e61ab45a688b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:11.252875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-67ef3f71-fec5-4a11-a43c-e61ab45a688b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:11.339415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:11.379464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef3f71-fec5-4a11-a43c-e61ab45a688b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:11.381045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 898/1783] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:11.412525 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c130c5-8bbc-44c3-955d-06ee46603331 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:11.414022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-70c130c5-8bbc-44c3-955d-06ee46603331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:11.511605 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a38f51-cc6e-4a40-9d9f-ca5498a36232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:11.512237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a38f51-cc6e-4a40-9d9f-ca5498a36232 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:11.512512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a38f51-cc6e-4a40-9d9f-ca5498a36232 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:11.512837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a38f51-cc6e-4a40-9d9f-ca5498a36232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:11.539943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c130c5-8bbc-44c3-955d-06ee46603331 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:11.544537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732a5f7-421c-4f18-b7e1-4f5d027531bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:11.545445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732a5f7-421c-4f18-b7e1-4f5d027531bb service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:11.546104 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732a5f7-421c-4f18-b7e1-4f5d027531bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:11.546589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732a5f7-421c-4f18-b7e1-4f5d027531bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:11.557030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732a5f7-421c-4f18-b7e1-4f5d027531bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:11.558815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732a5f7-421c-4f18-b7e1-4f5d027531bb service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:11.583870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 899/1784] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:11.613908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a38f51-cc6e-4a40-9d9f-ca5498a36232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:11.615721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a38f51-cc6e-4a40-9d9f-ca5498a36232 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:11.657967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 886/1785] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:11.731776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732a5f7-421c-4f18-b7e1-4f5d027531bb service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:11.736370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8732a5f7-421c-4f18-b7e1-4f5d027531bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:11.800635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732a5f7-421c-4f18-b7e1-4f5d027531bb service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:11.827074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:11.833392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:11.836564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:11.838382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] PATH_INFO: `/v3/users/b77d9421d0fd4eb899fc8bb133ff10ff` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:11.842329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b77d9421d0fd4eb899fc8bb133ff10ff)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:11.847132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:11.973308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b77d9421d0fd4eb899fc8bb133ff10ff'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:11.975239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b77d9421d0fd4eb899fc8bb133ff10ff'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:11.988776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:11.993628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b77d9421d0fd4eb899fc8bb133ff10ff'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:11.997275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b77d9421d0fd4eb899fc8bb133ff10ff'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:12.020853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2a9898c5-9b14-48db-960d-79e508cc1bad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b77d9421d0fd4eb899fc8bb133ff10ff'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:12.082573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 900/1786] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:11 2022] DELETE /identity/v3/users/b77d9421d0fd4eb899fc8bb133ff10ff => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:12.084237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32583e27-5ad5-4373-afbe-c554ddcb0998 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:12.084689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32583e27-5ad5-4373-afbe-c554ddcb0998 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:12.085182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32583e27-5ad5-4373-afbe-c554ddcb0998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:12.177406 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788378d-1d74-4120-b7a2-241df2e51e2b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:12.178143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788378d-1d74-4120-b7a2-241df2e51e2b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:12.178890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788378d-1d74-4120-b7a2-241df2e51e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:12.245607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a788378d-1d74-4120-b7a2-241df2e51e2b None None] MFA Rules not processed for user `8f5a88f263014f15a6f59cfdaa2bb6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:12.249917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a788378d-1d74-4120-b7a2-241df2e51e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:12.255388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-32583e27-5ad5-4373-afbe-c554ddcb0998 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:12.264729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-32583e27-5ad5-4373-afbe-c554ddcb0998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:12.454091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 901/1787] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Oct 5 20:22:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:12.519709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788378d-1d74-4120-b7a2-241df2e51e2b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:12.525271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a788378d-1d74-4120-b7a2-241df2e51e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:12.601769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 887/1788] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:12.604842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788378d-1d74-4120-b7a2-241df2e51e2b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:12.611902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32583e27-5ad5-4373-afbe-c554ddcb0998 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:12.614316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-32583e27-5ad5-4373-afbe-c554ddcb0998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:12.626559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bf9ca-72f8-49e6-8ca1-81060437536b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:12.631990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bf9ca-72f8-49e6-8ca1-81060437536b service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:12.642721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32583e27-5ad5-4373-afbe-c554ddcb0998 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:12.643379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:12.644052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:12.644339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:12.644611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] PATH_INFO: `/v3/projects/c977c2d5d91445c2a431e8384a285387` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:12.645078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bf9ca-72f8-49e6-8ca1-81060437536b service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:12.645673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bf9ca-72f8-49e6-8ca1-81060437536b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:12.646039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c977c2d5d91445c2a431e8384a285387)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:12.672951 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bf9ca-72f8-49e6-8ca1-81060437536b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:12.675586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bf9ca-72f8-49e6-8ca1-81060437536b service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:12.679703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:12.744275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 902/1789] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:12.756298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bf9ca-72f8-49e6-8ca1-81060437536b service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:12.804263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bf9ca-72f8-49e6-8ca1-81060437536b service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:12.825564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0473a2-03c7-4f80-8077-04e149501cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:12.830103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0473a2-03c7-4f80-8077-04e149501cfd service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:12.833748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0473a2-03c7-4f80-8077-04e149501cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:12.834506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0473a2-03c7-4f80-8077-04e149501cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:12.847502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:12.851719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c977c2d5d91445c2a431e8384a285387', 'request_id': 'req-20175ae0-1185-435d-956a-1b2dc2b4833c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:12.874334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c977c2d5d91445c2a431e8384a285387', 'request_id': 'req-20175ae0-1185-435d-956a-1b2dc2b4833c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:12.874530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0473a2-03c7-4f80-8077-04e149501cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:12.884067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0473a2-03c7-4f80-8077-04e149501cfd service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:12.901545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c977c2d5d91445c2a431e8384a285387', 'request_id': 'req-20175ae0-1185-435d-956a-1b2dc2b4833c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:12.922987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 903/1790] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:12.927613 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0473a2-03c7-4f80-8077-04e149501cfd service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:12.935107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1f0473a2-03c7-4f80-8077-04e149501cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:12.960044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] The token cache is being invalidate because project c977c2d5d91445c2a431e8384a285387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:12.963166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:12.969452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:12.975396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 888/1791] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:12 2022] DELETE /identity/v3/projects/c977c2d5d91445c2a431e8384a285387 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:12.979104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:12.990982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:13.030584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0473a2-03c7-4f80-8077-04e149501cfd service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:13.239117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3f368-258f-43b8-8a55-c01850374719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:13.239892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3f368-258f-43b8-8a55-c01850374719 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:13.240023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3f368-258f-43b8-8a55-c01850374719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:13.240274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3f368-258f-43b8-8a55-c01850374719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:13.244735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ded3f368-258f-43b8-8a55-c01850374719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:13.252076 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20175ae0-1185-435d-956a-1b2dc2b4833c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:13.264676 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:13.265339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:13.265645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:13.265923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] PATH_INFO: `/v3/users/d65c6109416f4ceca10ec0ebdb30cf71` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:13.268356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d65c6109416f4ceca10ec0ebdb30cf71)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:13.270785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:13.271809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3f368-258f-43b8-8a55-c01850374719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:13.275518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3f368-258f-43b8-8a55-c01850374719 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:13.317626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 904/1792] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:13.441194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd65c6109416f4ceca10ec0ebdb30cf71'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:13.443719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd65c6109416f4ceca10ec0ebdb30cf71'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:13.466407 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd65c6109416f4ceca10ec0ebdb30cf71'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:13.484863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd65c6109416f4ceca10ec0ebdb30cf71'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:13.486175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2ff440f-036b-4098-8bed-ba8c7d0a3831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd65c6109416f4ceca10ec0ebdb30cf71'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:13.543800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 889/1793] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:12 2022] DELETE /identity/v3/users/d65c6109416f4ceca10ec0ebdb30cf71 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:15.211729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3f368-258f-43b8-8a55-c01850374719 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:15.254421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3f368-258f-43b8-8a55-c01850374719 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:15.388942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:15.398563 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:15.399447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:15.399562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:15.399775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] PATH_INFO: `/v3/projects/d39bb953cefb4cee865efe783de6d2f9` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:15.405470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d39bb953cefb4cee865efe783de6d2f9)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:15.406742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:15.512844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd39bb953cefb4cee865efe783de6d2f9', 'request_id': 'req-c6e2a467-59ab-412c-b201-fb5b72a645f8'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:15.529501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd39bb953cefb4cee865efe783de6d2f9', 'request_id': 'req-c6e2a467-59ab-412c-b201-fb5b72a645f8'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:15.552129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd39bb953cefb4cee865efe783de6d2f9', 'request_id': 'req-c6e2a467-59ab-412c-b201-fb5b72a645f8'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:15.613946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] The token cache is being invalidate because project d39bb953cefb4cee865efe783de6d2f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:15.614798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:15.617153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:15.619452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 905/1794] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:15 2022] DELETE /identity/v3/projects/d39bb953cefb4cee865efe783de6d2f9 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:15.624144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ed70-d5a2-4f12-a74a-51a0a732fd9e None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:15.624649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ed70-d5a2-4f12-a74a-51a0a732fd9e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:15.625304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ed70-d5a2-4f12-a74a-51a0a732fd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:15.692174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:15.694085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:15.698542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-2f93ed70-d5a2-4f12-a74a-51a0a732fd9e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:15.701885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2f93ed70-d5a2-4f12-a74a-51a0a732fd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:15.869689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:15.909086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 890/1795] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:15.925494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fec721-842e-4db1-8d61-1d9efbac4eca None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:15.926629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fec721-842e-4db1-8d61-1d9efbac4eca None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:15.927466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fec721-842e-4db1-8d61-1d9efbac4eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:15.957642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2a467-59ab-412c-b201-fb5b72a645f8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:15.961626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-d2fec721-842e-4db1-8d61-1d9efbac4eca None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:15.963267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d2fec721-842e-4db1-8d61-1d9efbac4eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:15.972155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:15.972813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:15.973091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:15.973414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] PATH_INFO: `/v3/projects/288266d288794c67a86a1c3377e5cafa` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:15.974693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 891/1796] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:15.975352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] RBAC: Authorizing `identity:delete_project(project_id=288266d288794c67a86a1c3377e5cafa)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:15.977728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:15.979221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fec721-842e-4db1-8d61-1d9efbac4eca None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:15.992424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fec721-842e-4db1-8d61-1d9efbac4eca None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:16.020969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f1f3c-e774-4ba8-ba21-522fc4838ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:16.022233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f1f3c-e774-4ba8-ba21-522fc4838ccd admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.023983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f1f3c-e774-4ba8-ba21-522fc4838ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.024348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f1f3c-e774-4ba8-ba21-522fc4838ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.026374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f1f3c-e774-4ba8-ba21-522fc4838ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:16.028501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f1f3c-e774-4ba8-ba21-522fc4838ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:16.033568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f1f3c-e774-4ba8-ba21-522fc4838ccd admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:16.067211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '288266d288794c67a86a1c3377e5cafa', 'request_id': 'req-349ade4e-acb0-47a9-93a6-f8393175a53c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:16.073554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 892/1797] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:22:16.078671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f1f3c-e774-4ba8-ba21-522fc4838ccd admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:16.081741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '288266d288794c67a86a1c3377e5cafa', 'request_id': 'req-349ade4e-acb0-47a9-93a6-f8393175a53c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:16.091362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '288266d288794c67a86a1c3377e5cafa', 'request_id': 'req-349ade4e-acb0-47a9-93a6-f8393175a53c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:16.101900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f1f3c-e774-4ba8-ba21-522fc4838ccd admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:16.108321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] The token cache is being invalidate because project 288266d288794c67a86a1c3377e5cafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:16.108994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:16.110208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-349ade4e-acb0-47a9-93a6-f8393175a53c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:16.112868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 906/1798] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:15 2022] DELETE /identity/v3/projects/288266d288794c67a86a1c3377e5cafa => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:16.118241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f13030-d1e9-4a90-8f4f-4a26775a3100 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.118971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f13030-d1e9-4a90-8f4f-4a26775a3100 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.119394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f13030-d1e9-4a90-8f4f-4a26775a3100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.154233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-37f13030-d1e9-4a90-8f4f-4a26775a3100 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:16.155803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-37f13030-d1e9-4a90-8f4f-4a26775a3100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:16.181545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:16.234011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07770dd-0b6a-4a25-b393-8cc4a55f0e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:16.236381 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07770dd-0b6a-4a25-b393-8cc4a55f0e31 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.236661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07770dd-0b6a-4a25-b393-8cc4a55f0e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.236929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07770dd-0b6a-4a25-b393-8cc4a55f0e31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.237583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07770dd-0b6a-4a25-b393-8cc4a55f0e31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:16.245577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07770dd-0b6a-4a25-b393-8cc4a55f0e31 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:16.255063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 907/1799] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:16.256707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c819284-935f-45e5-af77-edda7fee7be7 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.257171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c819284-935f-45e5-af77-edda7fee7be7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.258424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c819284-935f-45e5-af77-edda7fee7be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.303037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-4c819284-935f-45e5-af77-edda7fee7be7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:16.304834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4c819284-935f-45e5-af77-edda7fee7be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:16.325483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 908/1800] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:16.329723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 893/1801] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:16 2022] POST /identity/v3/projects => generated 381 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:16.330200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c819284-935f-45e5-af77-edda7fee7be7 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:16.333188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07770dd-0b6a-4a25-b393-8cc4a55f0e31 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:16.335215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a07770dd-0b6a-4a25-b393-8cc4a55f0e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:16.360177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c819284-935f-45e5-af77-edda7fee7be7 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:16.461747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07770dd-0b6a-4a25-b393-8cc4a55f0e31 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:16.466966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352bc6-2e58-4462-a756-b7f820f2c9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:16.471089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352bc6-2e58-4462-a756-b7f820f2c9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.471840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352bc6-2e58-4462-a756-b7f820f2c9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.472294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352bc6-2e58-4462-a756-b7f820f2c9a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.473127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352bc6-2e58-4462-a756-b7f820f2c9a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:16.475466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352bc6-2e58-4462-a756-b7f820f2c9a6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:16.522563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 894/1802] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:16 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:16.525708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352bc6-2e58-4462-a756-b7f820f2c9a6 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:16.546275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba2d2a-a028-4b94-8f52-fd196f33cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:16.546916 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba2d2a-a028-4b94-8f52-fd196f33cbce admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.547209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba2d2a-a028-4b94-8f52-fd196f33cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.547464 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba2d2a-a028-4b94-8f52-fd196f33cbce admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.548127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba2d2a-a028-4b94-8f52-fd196f33cbce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:16.548487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba2d2a-a028-4b94-8f52-fd196f33cbce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:16.550150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba2d2a-a028-4b94-8f52-fd196f33cbce admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:16.565813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352bc6-2e58-4462-a756-b7f820f2c9a6 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:16.581193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8311a-de9e-493c-915e-326252ab02c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:16.582227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8311a-de9e-493c-915e-326252ab02c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.582729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8311a-de9e-493c-915e-326252ab02c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.583308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8311a-de9e-493c-915e-326252ab02c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.592119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 909/1803] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:22:16.594611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8311a-de9e-493c-915e-326252ab02c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:16.598016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba2d2a-a028-4b94-8f52-fd196f33cbce admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:16.600154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b8311a-de9e-493c-915e-326252ab02c4 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:16.633499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 895/1804] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:16.649129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8311a-de9e-493c-915e-326252ab02c4 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:16.661317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba2d2a-a028-4b94-8f52-fd196f33cbce admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:16.680475 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aef1e5-99de-4f4f-a5b3-ad6307d81c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:16.681190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aef1e5-99de-4f4f-a5b3-ad6307d81c7f admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.681469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aef1e5-99de-4f4f-a5b3-ad6307d81c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.681770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aef1e5-99de-4f4f-a5b3-ad6307d81c7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.682415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aef1e5-99de-4f4f-a5b3-ad6307d81c7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:16.693447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8311a-de9e-493c-915e-326252ab02c4 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:16.694917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aef1e5-99de-4f4f-a5b3-ad6307d81c7f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:16.713688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62d526-d427-4bb7-bb46-c730b88d3f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:16.714455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62d526-d427-4bb7-bb46-c730b88d3f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.714742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62d526-d427-4bb7-bb46-c730b88d3f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.715016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62d526-d427-4bb7-bb46-c730b88d3f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.715664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62d526-d427-4bb7-bb46-c730b88d3f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:16.717824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62d526-d427-4bb7-bb46-c730b88d3f58 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:16.750761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 896/1805] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:16.754833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62d526-d427-4bb7-bb46-c730b88d3f58 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:16.775194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62d526-d427-4bb7-bb46-c730b88d3f58 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:16.806733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f9f1e-8ee2-41c0-954f-27ccbb75d136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:16.807552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f9f1e-8ee2-41c0-954f-27ccbb75d136 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.808690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f9f1e-8ee2-41c0-954f-27ccbb75d136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.809092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f9f1e-8ee2-41c0-954f-27ccbb75d136 admin admin] PATH_INFO: `/v3/projects/8ffd874b3f4e4396a469e187c7bbd1b5/users/8bc341bb76c043c183b19c8f27b1bf0c/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.827942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 910/1806] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:16 2022] POST /identity/v3/projects => generated 359 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:16.828592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aef1e5-99de-4f4f-a5b3-ad6307d81c7f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:16.889991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f9f1e-8ee2-41c0-954f-27ccbb75d136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffd874b3f4e4396a469e187c7bbd1b5, user_id=8bc341bb76c043c183b19c8f27b1bf0c, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:16.893578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f9f1e-8ee2-41c0-954f-27ccbb75d136 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:16.956501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 897/1807] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:16 2022] PUT /identity/v3/projects/8ffd874b3f4e4396a469e187c7bbd1b5/users/8bc341bb76c043c183b19c8f27b1bf0c/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:16.961269 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacb291-17b2-47f5-834b-64f665f1f78d None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:16.961442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacb291-17b2-47f5-834b-64f665f1f78d None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:16.961735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacb291-17b2-47f5-834b-64f665f1f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:16.996097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:17.001989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aef1e5-99de-4f4f-a5b3-ad6307d81c7f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.055906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:17.076079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-bfacb291-17b2-47f5-834b-64f665f1f78d None None] MFA Rules not processed for user `8bc341bb76c043c183b19c8f27b1bf0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:17.077404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-bfacb291-17b2-47f5-834b-64f665f1f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:17.139788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc489d8-2058-4e0a-b794-c19b7ce7dbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.144142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc489d8-2058-4e0a-b794-c19b7ce7dbbd admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.144427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc489d8-2058-4e0a-b794-c19b7ce7dbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.144703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc489d8-2058-4e0a-b794-c19b7ce7dbbd admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.145902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc489d8-2058-4e0a-b794-c19b7ce7dbbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.151578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc489d8-2058-4e0a-b794-c19b7ce7dbbd admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.198069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 898/1808] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:17.202651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 911/1809] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:16 2022] POST /identity/v3/users => generated 280 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:17.203364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacb291-17b2-47f5-834b-64f665f1f78d None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.203720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc489d8-2058-4e0a-b794-c19b7ce7dbbd admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.219611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc489d8-2058-4e0a-b794-c19b7ce7dbbd admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.220729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacb291-17b2-47f5-834b-64f665f1f78d None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.226096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b97be-f319-4976-8590-961524472cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.226345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ce933-3ee1-4bf6-b3c7-f06245e5fda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.226910 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b97be-f319-4976-8590-961524472cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.227266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ce933-3ee1-4bf6-b3c7-f06245e5fda4 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.227377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b97be-f319-4976-8590-961524472cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.227716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b97be-f319-4976-8590-961524472cdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.227966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ce933-3ee1-4bf6-b3c7-f06245e5fda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.228486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ce933-3ee1-4bf6-b3c7-f06245e5fda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.228595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b97be-f319-4976-8590-961524472cdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.229394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ce933-3ee1-4bf6-b3c7-f06245e5fda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.231181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ce933-3ee1-4bf6-b3c7-f06245e5fda4 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.232840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b97be-f319-4976-8590-961524472cdb admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.253014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 912/1810] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:17.255108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ce933-3ee1-4bf6-b3c7-f06245e5fda4 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.270909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ce933-3ee1-4bf6-b3c7-f06245e5fda4 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.277588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fee3b0-66ed-4a20-a7f6-e9295714f29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.278331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee3b0-66ed-4a20-a7f6-e9295714f29e admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.278456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 899/1811] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:17 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:17.278628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee3b0-66ed-4a20-a7f6-e9295714f29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.279506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fee3b0-66ed-4a20-a7f6-e9295714f29e admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.280192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fee3b0-66ed-4a20-a7f6-e9295714f29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.281833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fee3b0-66ed-4a20-a7f6-e9295714f29e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.282730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b97be-f319-4976-8590-961524472cdb admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.299668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 913/1812] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:17.300769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fee3b0-66ed-4a20-a7f6-e9295714f29e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.317199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fee3b0-66ed-4a20-a7f6-e9295714f29e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.397871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b97be-f319-4976-8590-961524472cdb admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.410913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d2787-cf61-44a6-acff-1c5cb27eb2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.411557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fcba4-7437-4a21-b969-437987aba565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.412353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fcba4-7437-4a21-b969-437987aba565 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.412715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fcba4-7437-4a21-b969-437987aba565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.413085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fcba4-7437-4a21-b969-437987aba565 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.413403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d2787-cf61-44a6-acff-1c5cb27eb2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.413826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d2787-cf61-44a6-acff-1c5cb27eb2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.414012 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fcba4-7437-4a21-b969-437987aba565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.414632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d2787-cf61-44a6-acff-1c5cb27eb2dc admin admin] PATH_INFO: `/v3/projects/5e00a75fae8845658a30844957ef0e06/users/28c5947df3e74b68a37915b80c6e11b3/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.416194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fcba4-7437-4a21-b969-437987aba565 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.435841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d2787-cf61-44a6-acff-1c5cb27eb2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e00a75fae8845658a30844957ef0e06, user_id=28c5947df3e74b68a37915b80c6e11b3, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.439746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d2787-cf61-44a6-acff-1c5cb27eb2dc admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.449548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 900/1813] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:17 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:22:17.451621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fc283-3c3b-4ae2-b718-e0cd8c80635f None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.451621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fc283-3c3b-4ae2-b718-e0cd8c80635f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.451918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fc283-3c3b-4ae2-b718-e0cd8c80635f None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.453769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 901/1814] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:22:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:22:17.454779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fc283-3c3b-4ae2-b718-e0cd8c80635f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.461936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 914/1815] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:17 2022] PUT /identity/v3/projects/5e00a75fae8845658a30844957ef0e06/users/28c5947df3e74b68a37915b80c6e11b3/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:17.463033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb8ce7-c895-41cf-bbbb-cb2997b40f70 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.463465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb8ce7-c895-41cf-bbbb-cb2997b40f70 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.463988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb8ce7-c895-41cf-bbbb-cb2997b40f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.485874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d9fb8ce7-c895-41cf-bbbb-cb2997b40f70 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:17.486795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d9fb8ce7-c895-41cf-bbbb-cb2997b40f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:17.557663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fc283-3c3b-4ae2-b718-e0cd8c80635f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.566293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d32d89-2589-4ae3-8a8c-d32471faad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.567304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d32d89-2589-4ae3-8a8c-d32471faad47 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.567581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d32d89-2589-4ae3-8a8c-d32471faad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.567848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d32d89-2589-4ae3-8a8c-d32471faad47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.569554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d32d89-2589-4ae3-8a8c-d32471faad47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.572160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d32d89-2589-4ae3-8a8c-d32471faad47 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.573721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:17.585958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 915/1816] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:22:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:17.587258 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb2bb-7ac0-4fc0-8e5c-c2bb2844e7e9 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.587540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb2bb-7ac0-4fc0-8e5c-c2bb2844e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.587848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb2bb-7ac0-4fc0-8e5c-c2bb2844e7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.600850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 902/1817] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:17.601987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d32d89-2589-4ae3-8a8c-d32471faad47 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.615069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d32d89-2589-4ae3-8a8c-d32471faad47 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.619062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:17.619830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d79a7-3f64-47fb-91d2-c4a02bf7ffc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.620489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d79a7-3f64-47fb-91d2-c4a02bf7ffc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.620771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d79a7-3f64-47fb-91d2-c4a02bf7ffc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.621033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d79a7-3f64-47fb-91d2-c4a02bf7ffc4 admin admin] PATH_INFO: `/v3/projects/94cf1e5c77d440edb7578b2420397055/users/0ea54c452a7f4bbcae1b52807543ee9d/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.622916 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d38fb2bb-7ac0-4fc0-8e5c-c2bb2844e7e9 None None] MFA Rules not processed for user `28c5947df3e74b68a37915b80c6e11b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:17.625531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d38fb2bb-7ac0-4fc0-8e5c-c2bb2844e7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:17.637760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d79a7-3f64-47fb-91d2-c4a02bf7ffc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cf1e5c77d440edb7578b2420397055, user_id=0ea54c452a7f4bbcae1b52807543ee9d, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.641572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d79a7-3f64-47fb-91d2-c4a02bf7ffc4 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.665911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 903/1818] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:17 2022] PUT /identity/v3/projects/94cf1e5c77d440edb7578b2420397055/users/0ea54c452a7f4bbcae1b52807543ee9d/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:17.666895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d79a7-3f64-47fb-91d2-c4a02bf7ffc4 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.668754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ea0d79a7-3f64-47fb-91d2-c4a02bf7ffc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:17.705894 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 916/1819] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:17.706745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fb2bb-7ac0-4fc0-8e5c-c2bb2844e7e9 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.721587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fb2bb-7ac0-4fc0-8e5c-c2bb2844e7e9 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.774788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d79a7-3f64-47fb-91d2-c4a02bf7ffc4 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.781264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c35bef-6823-4537-8d9a-a1a174da6067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.782008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c35bef-6823-4537-8d9a-a1a174da6067 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.782516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c35bef-6823-4537-8d9a-a1a174da6067 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.782797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c35bef-6823-4537-8d9a-a1a174da6067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.788254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9facc-8a19-4be8-8cee-63b6ab58c701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.788868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9facc-8a19-4be8-8cee-63b6ab58c701 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.789571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9facc-8a19-4be8-8cee-63b6ab58c701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.789571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9facc-8a19-4be8-8cee-63b6ab58c701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.790072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9facc-8a19-4be8-8cee-63b6ab58c701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.791790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9facc-8a19-4be8-8cee-63b6ab58c701 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.792465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c35bef-6823-4537-8d9a-a1a174da6067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.794237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c35bef-6823-4537-8d9a-a1a174da6067 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.801868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 917/1820] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:17.805166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 904/1821] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:17.805613 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9facc-8a19-4be8-8cee-63b6ab58c701 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.807308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b1b9facc-8a19-4be8-8cee-63b6ab58c701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:17.807685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c35bef-6823-4537-8d9a-a1a174da6067 service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:17.819058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c35bef-6823-4537-8d9a-a1a174da6067 service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.824307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abf934-7a4a-4d96-9c33-955dac15bb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.824967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abf934-7a4a-4d96-9c33-955dac15bb13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.825285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abf934-7a4a-4d96-9c33-955dac15bb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.826151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abf934-7a4a-4d96-9c33-955dac15bb13 admin admin] PATH_INFO: `/v3/projects/94cf1e5c77d440edb7578b2420397055/users/0ea54c452a7f4bbcae1b52807543ee9d/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.828180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abf934-7a4a-4d96-9c33-955dac15bb13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cf1e5c77d440edb7578b2420397055, user_id=0ea54c452a7f4bbcae1b52807543ee9d, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:17.830845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abf934-7a4a-4d96-9c33-955dac15bb13 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:17.845785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 905/1822] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:17 2022] PUT /identity/v3/projects/94cf1e5c77d440edb7578b2420397055/users/0ea54c452a7f4bbcae1b52807543ee9d/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:17.858060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344601b-aa5f-41b4-86a9-b62b7639866c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.858347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344601b-aa5f-41b4-86a9-b62b7639866c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.858617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344601b-aa5f-41b4-86a9-b62b7639866c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.887914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9facc-8a19-4be8-8cee-63b6ab58c701 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:17.905115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e344601b-aa5f-41b4-86a9-b62b7639866c None None] MFA Rules not processed for user `0ea54c452a7f4bbcae1b52807543ee9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:17.906478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e344601b-aa5f-41b4-86a9-b62b7639866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:17.941449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a4f61-a383-437a-b728-354ad0ebefd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:17.942155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a4f61-a383-437a-b728-354ad0ebefd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:17.942424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a4f61-a383-437a-b728-354ad0ebefd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:17.942694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a4f61-a383-437a-b728-354ad0ebefd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:17.952374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:17.989974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 906/1823] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:18.002940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344601b-aa5f-41b4-86a9-b62b7639866c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:18.004484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e344601b-aa5f-41b4-86a9-b62b7639866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:18.011961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a4f61-a383-437a-b728-354ad0ebefd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:18.014286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a4f61-a383-437a-b728-354ad0ebefd0 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:18.016131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344601b-aa5f-41b4-86a9-b62b7639866c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:18.020420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd4ca2-0e4e-4ce9-bbab-608218d76acf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:18.021034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4ca2-0e4e-4ce9-bbab-608218d76acf service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:18.021314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4ca2-0e4e-4ce9-bbab-608218d76acf service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:18.021697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4ca2-0e4e-4ce9-bbab-608218d76acf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:18.023429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 918/1824] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:18.031393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd4ca2-0e4e-4ce9-bbab-608218d76acf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:18.033065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd4ca2-0e4e-4ce9-bbab-608218d76acf service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:18.042588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a4f61-a383-437a-b728-354ad0ebefd0 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:18.053565 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 907/1825] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:18.130976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a4f61-a383-437a-b728-354ad0ebefd0 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:18.136163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:18.136876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:18.137086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:18.137335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] PATH_INFO: `/v3/users/28c5947df3e74b68a37915b80c6e11b3` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:18.138535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c5947df3e74b68a37915b80c6e11b3)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:18.140453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:18.245973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c5947df3e74b68a37915b80c6e11b3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:18.248651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c5947df3e74b68a37915b80c6e11b3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:18.261087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c5947df3e74b68a37915b80c6e11b3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:18.267583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c5947df3e74b68a37915b80c6e11b3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:18.289469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c5947df3e74b68a37915b80c6e11b3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:18.338446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 919/1826] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:18 2022] DELETE /identity/v3/users/28c5947df3e74b68a37915b80c6e11b3 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:18.343780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867705a-f238-407f-b654-85b098b67a8a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:18.344252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867705a-f238-407f-b654-85b098b67a8a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:18.344604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867705a-f238-407f-b654-85b098b67a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:18.396989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9867705a-f238-407f-b654-85b098b67a8a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:18.402122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9867705a-f238-407f-b654-85b098b67a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:18.433051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:18.576586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 908/1827] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:18.587931 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:18.589327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:18.650955 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:18.724766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790b69f-6dac-4d7e-a373-5814b38197eb admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:18.728941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39afae-bd55-411d-b372-b43e8be74f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:18.729596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39afae-bd55-411d-b372-b43e8be74f91 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:18.729880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39afae-bd55-411d-b372-b43e8be74f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:18.730275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39afae-bd55-411d-b372-b43e8be74f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:18.741023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39afae-bd55-411d-b372-b43e8be74f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:18.742519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39afae-bd55-411d-b372-b43e8be74f91 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:18.755047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 920/1828] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:18.964827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867705a-f238-407f-b654-85b098b67a8a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:18.967121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9867705a-f238-407f-b654-85b098b67a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:19.196136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867705a-f238-407f-b654-85b098b67a8a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:19.211816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c7f92-a06e-4e2a-a9b5-198ae4285cb2 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea54c452a7f4bbcae1b52807543ee9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94cf1e5c77d440edb7578b2420397055', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:19.212464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c7f92-a06e-4e2a-a9b5-198ae4285cb2 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:19.212749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c7f92-a06e-4e2a-a9b5-198ae4285cb2 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:19.213027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c7f92-a06e-4e2a-a9b5-198ae4285cb2 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:19.221183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c7f92-a06e-4e2a-a9b5-198ae4285cb2 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:19.221684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c7f92-a06e-4e2a-a9b5-198ae4285cb2 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:19.223180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c7f92-a06e-4e2a-a9b5-198ae4285cb2 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:19.255673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 909/1829] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:22:19.259737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39afae-bd55-411d-b372-b43e8be74f91 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:19.279632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39afae-bd55-411d-b372-b43e8be74f91 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:19.287628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37962a-c2ee-4658-8862-a6146726824d tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea54c452a7f4bbcae1b52807543ee9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94cf1e5c77d440edb7578b2420397055', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:19.288338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37962a-c2ee-4658-8862-a6146726824d tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:19.288562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37962a-c2ee-4658-8862-a6146726824d tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:19.288836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37962a-c2ee-4658-8862-a6146726824d tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:19.289544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37962a-c2ee-4658-8862-a6146726824d tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:19.292487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37962a-c2ee-4658-8862-a6146726824d tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:19.352334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 921/1830] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:19 2022] POST /identity/v3/projects => generated 354 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:19.356482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c7f92-a06e-4e2a-a9b5-198ae4285cb2 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:19.375225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c7f92-a06e-4e2a-a9b5-198ae4285cb2 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:19.616884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea54c452a7f4bbcae1b52807543ee9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94cf1e5c77d440edb7578b2420397055', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:19.617962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:19.618268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:19.618652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:19.620346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:19.622047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:19.626151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:19.631087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37962a-c2ee-4658-8862-a6146726824d tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:19.633050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7e37962a-c2ee-4658-8862-a6146726824d tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:19.673320 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 910/1831] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:22:19.707181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:19.709890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:19.807628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37962a-c2ee-4658-8862-a6146726824d tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:19.813645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:19.814571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:19.815087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:19.815550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] PATH_INFO: `/v3/users/8baad395c74d4c7196960010a98fd5f2` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:19.827889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8baad395c74d4c7196960010a98fd5f2)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:19.832945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:19.864878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:19.930930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8baad395c74d4c7196960010a98fd5f2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:19.940646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8baad395c74d4c7196960010a98fd5f2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:19.949821 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8baad395c74d4c7196960010a98fd5f2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:19.950844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8baad395c74d4c7196960010a98fd5f2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:19.970860 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8baad395c74d4c7196960010a98fd5f2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:19.971038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327f6f4-e5cd-4cf1-8a62-2f9a1eeb61b0 tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:19.990380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:19.990380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:19.990380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:19.990380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] PATH_INFO: `/v3/projects/5e00a75fae8845658a30844957ef0e06` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:19.993715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e00a75fae8845658a30844957ef0e06)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:19.995647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:20.048616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 922/1832] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:19 2022] DELETE /identity/v3/users/8baad395c74d4c7196960010a98fd5f2 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:20.049479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:20.052614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:20.178396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e00a75fae8845658a30844957ef0e06', 'request_id': 'req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.196337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e00a75fae8845658a30844957ef0e06', 'request_id': 'req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.202180 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:20.211420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e00a75fae8845658a30844957ef0e06', 'request_id': 'req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.264495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] The token cache is being invalidate because project 5e00a75fae8845658a30844957ef0e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:20.265206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.267510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-45f4dc7b-05d9-44f0-805d-7d7b5fe9b455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.270211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 911/1833] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:19 2022] DELETE /identity/v3/projects/5e00a75fae8845658a30844957ef0e06 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:20.272449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367658-1a27-4caf-9043-6792d8700744 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:20.273051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367658-1a27-4caf-9043-6792d8700744 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:20.273623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52367658-1a27-4caf-9043-6792d8700744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:20.321256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-52367658-1a27-4caf-9043-6792d8700744 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:20.322877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-52367658-1a27-4caf-9043-6792d8700744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:20.328185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cffec9-d990-4a51-8ac3-f0c73bd7f863 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:20.482734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ea54c452a7f4bbcae1b52807543ee9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94cf1e5c77d440edb7578b2420397055', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:20.484390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:20.484680 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:20.484954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] PATH_INFO: `/v3/projects/19199d1da05c4c73a59c21458b129d2c` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:20.488321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Authorizing `identity:delete_project(project_id=19199d1da05c4c73a59c21458b129d2c)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:20.492899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:20.496622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 912/1834] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:20.500984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f36134-8b5e-4a6f-b450-d1e493e94207 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:20.503150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f36134-8b5e-4a6f-b450-d1e493e94207 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:20.504715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f36134-8b5e-4a6f-b450-d1e493e94207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:20.588718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a1f36134-8b5e-4a6f-b450-d1e493e94207 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:20.589834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a1f36134-8b5e-4a6f-b450-d1e493e94207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:20.632280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19199d1da05c4c73a59c21458b129d2c', 'request_id': 'req-6794624e-40b4-4a94-abf1-94e16ef03dec'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.657737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19199d1da05c4c73a59c21458b129d2c', 'request_id': 'req-6794624e-40b4-4a94-abf1-94e16ef03dec'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.668435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19199d1da05c4c73a59c21458b129d2c', 'request_id': 'req-6794624e-40b4-4a94-abf1-94e16ef03dec'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.696090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] The token cache is being invalidate because project 19199d1da05c4c73a59c21458b129d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:20.696465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.704625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:20.712631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 923/1835] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:20 2022] DELETE /identity/v3/projects/19199d1da05c4c73a59c21458b129d2c => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:20.731586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:20.733851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:20.808605 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 913/1836] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:20.817376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd6669-b5b1-4cb6-8c6b-795e98371079 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:20.817748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd6669-b5b1-4cb6-8c6b-795e98371079 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:20.818593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd6669-b5b1-4cb6-8c6b-795e98371079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:20.848960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794624e-40b4-4a94-abf1-94e16ef03dec tempest-VolumeQuotaClassesTest-903414764 tempest-VolumeQuotaClassesTest-903414764-project-admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:20.887232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-abbd6669-b5b1-4cb6-8c6b-795e98371079 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:20.890287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-abbd6669-b5b1-4cb6-8c6b-795e98371079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:20.898731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a35d70-6016-49ac-b0c0-3122fd7353d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:20.899386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a35d70-6016-49ac-b0c0-3122fd7353d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:20.899664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a35d70-6016-49ac-b0c0-3122fd7353d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:20.899953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a35d70-6016-49ac-b0c0-3122fd7353d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:20.916482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-65a35d70-6016-49ac-b0c0-3122fd7353d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:20.923305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 914/1837] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:20.925746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd6669-b5b1-4cb6-8c6b-795e98371079 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:20.966646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a35d70-6016-49ac-b0c0-3122fd7353d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:20.968044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a35d70-6016-49ac-b0c0-3122fd7353d7 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:20.975357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd6669-b5b1-4cb6-8c6b-795e98371079 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:20.980742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 924/1838] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:20.986240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952592c4-a1cd-4233-af6b-9f19a58dc1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:20.987030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952592c4-a1cd-4233-af6b-9f19a58dc1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:20.987385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952592c4-a1cd-4233-af6b-9f19a58dc1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:20.987719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952592c4-a1cd-4233-af6b-9f19a58dc1b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:20.988708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952592c4-a1cd-4233-af6b-9f19a58dc1b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:20.989084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952592c4-a1cd-4233-af6b-9f19a58dc1b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:20.990854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952592c4-a1cd-4233-af6b-9f19a58dc1b2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:20.993678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:21.014727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a35d70-6016-49ac-b0c0-3122fd7353d7 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:21.015779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 915/1839] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:22:21.025873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a35d70-6016-49ac-b0c0-3122fd7353d7 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:21.031756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a94e-446b-41c0-95a2-0d03d2ff22bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:21.032460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a94e-446b-41c0-95a2-0d03d2ff22bb admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:21.032813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a94e-446b-41c0-95a2-0d03d2ff22bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:21.033153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a94e-446b-41c0-95a2-0d03d2ff22bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:21.042224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2a94e-446b-41c0-95a2-0d03d2ff22bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:21.053287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2a94e-446b-41c0-95a2-0d03d2ff22bb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:21.063242 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:21.122462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 925/1840] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:21 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:21.130618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952592c4-a1cd-4233-af6b-9f19a58dc1b2 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:21.160890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952592c4-a1cd-4233-af6b-9f19a58dc1b2 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:21.313784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ab6e-fde1-4f07-b62f-4c75550bfc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:21.314419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ab6e-fde1-4f07-b62f-4c75550bfc63 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:21.314686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ab6e-fde1-4f07-b62f-4c75550bfc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:21.314953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ab6e-fde1-4f07-b62f-4c75550bfc63 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:21.315590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223ab6e-fde1-4f07-b62f-4c75550bfc63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:21.321930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223ab6e-fde1-4f07-b62f-4c75550bfc63 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:21.380688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 916/1841] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:21 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:21.382492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a94e-446b-41c0-95a2-0d03d2ff22bb admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:21.383662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ab6e-fde1-4f07-b62f-4c75550bfc63 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:21.385561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-32b2a94e-446b-41c0-95a2-0d03d2ff22bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:21.399101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ab6e-fde1-4f07-b62f-4c75550bfc63 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:21.413782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a94e-446b-41c0-95a2-0d03d2ff22bb admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:21.430544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:21.431799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:21.433922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:21.434367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] PATH_INFO: `/v3/users/8bc341bb76c043c183b19c8f27b1bf0c` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:21.438296 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc341bb76c043c183b19c8f27b1bf0c)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:21.441038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:21.458975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176bc09-34bb-4bfc-a7bd-b25ae97ac333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:21.460051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176bc09-34bb-4bfc-a7bd-b25ae97ac333 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:21.460512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176bc09-34bb-4bfc-a7bd-b25ae97ac333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:21.462143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176bc09-34bb-4bfc-a7bd-b25ae97ac333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:21.464532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176bc09-34bb-4bfc-a7bd-b25ae97ac333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:21.467333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176bc09-34bb-4bfc-a7bd-b25ae97ac333 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:21.495705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 917/1842] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:21.499857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176bc09-34bb-4bfc-a7bd-b25ae97ac333 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:21.528182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176bc09-34bb-4bfc-a7bd-b25ae97ac333 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:21.541170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc71622-e083-4675-bbf6-a3da56269b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:21.542409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc71622-e083-4675-bbf6-a3da56269b86 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:21.544094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc71622-e083-4675-bbf6-a3da56269b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:21.544649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc71622-e083-4675-bbf6-a3da56269b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:21.545620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc71622-e083-4675-bbf6-a3da56269b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:21.549968 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc71622-e083-4675-bbf6-a3da56269b86 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:21.561981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc341bb76c043c183b19c8f27b1bf0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:21.563767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc341bb76c043c183b19c8f27b1bf0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:21.585034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 918/1843] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:21.588829 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc71622-e083-4675-bbf6-a3da56269b86 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:21.607227 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc341bb76c043c183b19c8f27b1bf0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:21.610664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc341bb76c043c183b19c8f27b1bf0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:21.632288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc71622-e083-4675-bbf6-a3da56269b86 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:21.637660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-79df1948-2ed9-4807-bcbf-6c6ba192ee7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc341bb76c043c183b19c8f27b1bf0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:21.658260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd8e44-5ae8-4a32-89d0-9a4d26145e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:21.660390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd8e44-5ae8-4a32-89d0-9a4d26145e28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:21.661349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd8e44-5ae8-4a32-89d0-9a4d26145e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:21.661872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd8e44-5ae8-4a32-89d0-9a4d26145e28 admin admin] PATH_INFO: `/v3/projects/552e1d71116e47f4a40689a642cfc8f6/users/0455304776e442e7b8c401dad99d9015/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:21.704356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 926/1844] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:21 2022] DELETE /identity/v3/users/8bc341bb76c043c183b19c8f27b1bf0c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:21.707195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a984a-045b-4e3d-8b3b-a094624a23ac None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:21.707485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a984a-045b-4e3d-8b3b-a094624a23ac None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:21.707751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a984a-045b-4e3d-8b3b-a094624a23ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:21.722819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd8e44-5ae8-4a32-89d0-9a4d26145e28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=552e1d71116e47f4a40689a642cfc8f6, user_id=0455304776e442e7b8c401dad99d9015, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:21.728545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd8e44-5ae8-4a32-89d0-9a4d26145e28 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:21.774382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b98a984a-045b-4e3d-8b3b-a094624a23ac None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:21.775648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b98a984a-045b-4e3d-8b3b-a094624a23ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:21.786072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 919/1845] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:21 2022] PUT /identity/v3/projects/552e1d71116e47f4a40689a642cfc8f6/users/0455304776e442e7b8c401dad99d9015/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:21.790644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231acc2-885a-4748-9846-1f9f88c62576 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:21.791400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231acc2-885a-4748-9846-1f9f88c62576 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:21.791400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231acc2-885a-4748-9846-1f9f88c62576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:21.852084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-d231acc2-885a-4748-9846-1f9f88c62576 None None] MFA Rules not processed for user `0455304776e442e7b8c401dad99d9015`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:21.853354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d231acc2-885a-4748-9846-1f9f88c62576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:21.962289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 927/1846] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:21.983704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a984a-045b-4e3d-8b3b-a094624a23ac None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:21.985634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b98a984a-045b-4e3d-8b3b-a094624a23ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:22.113301 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 920/1847] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:22.135530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231acc2-885a-4748-9846-1f9f88c62576 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:22.146755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d231acc2-885a-4748-9846-1f9f88c62576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:22.176768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a984a-045b-4e3d-8b3b-a094624a23ac None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:22.182831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f26d9c-2232-443e-a7ae-2a1a246caf15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:22.183478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26d9c-2232-443e-a7ae-2a1a246caf15 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:22.183804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26d9c-2232-443e-a7ae-2a1a246caf15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:22.184080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26d9c-2232-443e-a7ae-2a1a246caf15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:22.187978 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231acc2-885a-4748-9846-1f9f88c62576 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:22.194174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f807c-27ec-488f-a49a-a4fb4626a14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:22.195224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f807c-27ec-488f-a49a-a4fb4626a14a admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:22.195861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f807c-27ec-488f-a49a-a4fb4626a14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:22.196377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f807c-27ec-488f-a49a-a4fb4626a14a admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:22.197549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f807c-27ec-488f-a49a-a4fb4626a14a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:22.200815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f807c-27ec-488f-a49a-a4fb4626a14a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:22.234505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f26d9c-2232-443e-a7ae-2a1a246caf15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:22.236162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f26d9c-2232-443e-a7ae-2a1a246caf15 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:22.243816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 921/1848] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:22 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:22.245128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f807c-27ec-488f-a49a-a4fb4626a14a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:22.258122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 928/1849] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:22.258722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:22.360643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f807c-27ec-488f-a49a-a4fb4626a14a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:22.365659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a2f6-28ac-4af7-896e-4ce4caf4682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:22.366277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a2f6-28ac-4af7-896e-4ce4caf4682c admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:22.366560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a2f6-28ac-4af7-896e-4ce4caf4682c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:22.366823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a2f6-28ac-4af7-896e-4ce4caf4682c admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:22.367464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a2f6-28ac-4af7-896e-4ce4caf4682c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:22.369419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a2f6-28ac-4af7-896e-4ce4caf4682c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:22.401538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 922/1850] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:22 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:22.405367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f26d9c-2232-443e-a7ae-2a1a246caf15 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:22.413289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:22.420422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f26d9c-2232-443e-a7ae-2a1a246caf15 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:22.427287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f287e70-f3df-4fb4-8a33-d731f67d78f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:22.427996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f287e70-f3df-4fb4-8a33-d731f67d78f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:22.429725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f287e70-f3df-4fb4-8a33-d731f67d78f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:22.430800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f287e70-f3df-4fb4-8a33-d731f67d78f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:22.431585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f287e70-f3df-4fb4-8a33-d731f67d78f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:22.433350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f287e70-f3df-4fb4-8a33-d731f67d78f1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:22.450201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a2f6-28ac-4af7-896e-4ce4caf4682c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:22.459040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 929/1851] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:22.465869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a2f6-28ac-4af7-896e-4ce4caf4682c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:22.474999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45fdb9-f416-4c95-92db-e3407f221767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:22.476201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45fdb9-f416-4c95-92db-e3407f221767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:22.476569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45fdb9-f416-4c95-92db-e3407f221767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:22.477463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45fdb9-f416-4c95-92db-e3407f221767 admin admin] PATH_INFO: `/v3/projects/9253fb977ae749d79b0a188dab556e0d/users/1aeed3165b0549cdb71921c3f0c054f6/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:22.503105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45fdb9-f416-4c95-92db-e3407f221767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9253fb977ae749d79b0a188dab556e0d, user_id=1aeed3165b0549cdb71921c3f0c054f6, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:22.523052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45fdb9-f416-4c95-92db-e3407f221767 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:22.566099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 923/1852] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:22 2022] PUT /identity/v3/projects/9253fb977ae749d79b0a188dab556e0d/users/1aeed3165b0549cdb71921c3f0c054f6/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:22.570672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f287e70-f3df-4fb4-8a33-d731f67d78f1 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:22.595770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f287e70-f3df-4fb4-8a33-d731f67d78f1 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:22.712282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d92208-f06a-402e-933f-e8e965e5b552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:22.712915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d92208-f06a-402e-933f-e8e965e5b552 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:22.713193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d92208-f06a-402e-933f-e8e965e5b552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:22.713477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d92208-f06a-402e-933f-e8e965e5b552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:22.714165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d92208-f06a-402e-933f-e8e965e5b552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:22.716497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d92208-f06a-402e-933f-e8e965e5b552 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:22.727544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 930/1853] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:22.734665 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d92208-f06a-402e-933f-e8e965e5b552 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:22.735057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45fdb9-f416-4c95-92db-e3407f221767 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:22.737285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-42d92208-f06a-402e-933f-e8e965e5b552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:22.753724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d92208-f06a-402e-933f-e8e965e5b552 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:22.761372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:22.762113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:22.762880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:22.763162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] PATH_INFO: `/v3/projects/caa4eefe582c4b55b199aece42a77ebe` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:22.764054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45fdb9-f416-4c95-92db-e3407f221767 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:22.764541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa4eefe582c4b55b199aece42a77ebe)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:22.767451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:22.781556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ec29c-6e1b-46c6-8539-27284ae093a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:22.783767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ec29c-6e1b-46c6-8539-27284ae093a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:22.784388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ec29c-6e1b-46c6-8539-27284ae093a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:22.785370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ec29c-6e1b-46c6-8539-27284ae093a2 admin admin] PATH_INFO: `/v3/projects/9253fb977ae749d79b0a188dab556e0d/users/1aeed3165b0549cdb71921c3f0c054f6/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:22.796548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ec29c-6e1b-46c6-8539-27284ae093a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9253fb977ae749d79b0a188dab556e0d, user_id=1aeed3165b0549cdb71921c3f0c054f6, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:22.802516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ec29c-6e1b-46c6-8539-27284ae093a2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:22.820038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 924/1854] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:22 2022] PUT /identity/v3/projects/9253fb977ae749d79b0a188dab556e0d/users/1aeed3165b0549cdb71921c3f0c054f6/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:22.832799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2003e9-e2db-4bca-b73c-eb21da0f1e36 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:22.833677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2003e9-e2db-4bca-b73c-eb21da0f1e36 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:22.834369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2003e9-e2db-4bca-b73c-eb21da0f1e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:22.863618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa4eefe582c4b55b199aece42a77ebe', 'request_id': 'req-87f670e0-8162-4d8a-8cdd-91b5f879d822'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:22.892505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa4eefe582c4b55b199aece42a77ebe', 'request_id': 'req-87f670e0-8162-4d8a-8cdd-91b5f879d822'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:22.900014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8a2003e9-e2db-4bca-b73c-eb21da0f1e36 None None] MFA Rules not processed for user `1aeed3165b0549cdb71921c3f0c054f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:22.902586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8a2003e9-e2db-4bca-b73c-eb21da0f1e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:22.905329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa4eefe582c4b55b199aece42a77ebe', 'request_id': 'req-87f670e0-8162-4d8a-8cdd-91b5f879d822'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:22.931732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] The token cache is being invalidate because project caa4eefe582c4b55b199aece42a77ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:22.932226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:22.932934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:22.939902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 931/1855] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:22 2022] DELETE /identity/v3/projects/caa4eefe582c4b55b199aece42a77ebe => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:22.945659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:22.952362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:23.110670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 925/1856] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:23.113053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818ed42-5ebf-457c-9561-96db83b4ebaf None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:23.113283 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818ed42-5ebf-457c-9561-96db83b4ebaf None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:23.113579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818ed42-5ebf-457c-9561-96db83b4ebaf None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:23.115238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 926/1857] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:22:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:22:23.115665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818ed42-5ebf-457c-9561-96db83b4ebaf None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:23.117343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5818ed42-5ebf-457c-9561-96db83b4ebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:23.142323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f670e0-8162-4d8a-8cdd-91b5f879d822 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:23.156278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:23.157191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:23.157740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:23.158557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] PATH_INFO: `/v3/users/a5315c3ac5a348d3b33065de79edf606` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:23.160056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5315c3ac5a348d3b33065de79edf606)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:23.162106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:23.188712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:23.199556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:23.204185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818ed42-5ebf-457c-9561-96db83b4ebaf None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:23.210267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:23.213094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:23.213453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:23.213932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:23.217894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:23.276748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5315c3ac5a348d3b33065de79edf606'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:23.277972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5315c3ac5a348d3b33065de79edf606'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:23.291965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5315c3ac5a348d3b33065de79edf606'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:23.292713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5315c3ac5a348d3b33065de79edf606'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:23.307874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c1b0abd6-985a-4178-9ad0-ebbdfb134627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5315c3ac5a348d3b33065de79edf606'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:23.329439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 932/1858] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:22 2022] DELETE /identity/v3/users/a5315c3ac5a348d3b33065de79edf606 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:23.330837 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b427f-9b0b-4b68-92a9-04ae2ecaf187 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:23.331123 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b427f-9b0b-4b68-92a9-04ae2ecaf187 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:23.331394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b427f-9b0b-4b68-92a9-04ae2ecaf187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:23.342893 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:23.344528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:23.354903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-bf6b427f-9b0b-4b68-92a9-04ae2ecaf187 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:23.356862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-bf6b427f-9b0b-4b68-92a9-04ae2ecaf187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:23.360970 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 927/1859] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:23.517992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 933/1860] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:22:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:23.672214 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:23.676033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:23.718149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b427f-9b0b-4b68-92a9-04ae2ecaf187 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:23.722127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-bf6b427f-9b0b-4b68-92a9-04ae2ecaf187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:23.883574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ad5ae-9ac6-4047-a2ce-106af3dcbc64 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:23.896296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a30bf1-fb8e-43b2-8c67-e93696ad75b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:23.896296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a30bf1-fb8e-43b2-8c67-e93696ad75b0 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:23.896296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a30bf1-fb8e-43b2-8c67-e93696ad75b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:23.896296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a30bf1-fb8e-43b2-8c67-e93696ad75b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:23.901851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a30bf1-fb8e-43b2-8c67-e93696ad75b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:23.903471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a30bf1-fb8e-43b2-8c67-e93696ad75b0 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:23.925958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 928/1861] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:23.939608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b427f-9b0b-4b68-92a9-04ae2ecaf187 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:23.953975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:23.954771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:23.956415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:23.957429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] PATH_INFO: `/v3/projects/8ffd874b3f4e4396a469e187c7bbd1b5` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:23.961621 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ffd874b3f4e4396a469e187c7bbd1b5)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:23.967885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:24.000792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:24.034130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ffd874b3f4e4396a469e187c7bbd1b5', 'request_id': 'req-73b5d1ce-72ba-4296-9210-d88070954abd'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.044553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ffd874b3f4e4396a469e187c7bbd1b5', 'request_id': 'req-73b5d1ce-72ba-4296-9210-d88070954abd'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.051330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ffd874b3f4e4396a469e187c7bbd1b5', 'request_id': 'req-73b5d1ce-72ba-4296-9210-d88070954abd'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.065233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] The token cache is being invalidate because project 8ffd874b3f4e4396a469e187c7bbd1b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:24.065483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.066295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.068551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 934/1862] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:23 2022] DELETE /identity/v3/projects/8ffd874b3f4e4396a469e187c7bbd1b5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:24.072700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a30bf1-fb8e-43b2-8c67-e93696ad75b0 service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:24.082321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:24.085297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:24.087284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a30bf1-fb8e-43b2-8c67-e93696ad75b0 service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:24.194624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:24.195291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.195555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.195818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] PATH_INFO: `/v3/users/0ea54c452a7f4bbcae1b52807543ee9d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.196964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea54c452a7f4bbcae1b52807543ee9d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:24.198923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:24.208263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5d1ce-72ba-4296-9210-d88070954abd admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:24.214096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:24.214910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.215185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.215463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] PATH_INFO: `/v3/projects/259508d5cb584fcda59e680e5222f1ff` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.217380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=259508d5cb584fcda59e680e5222f1ff)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:24.220196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:24.230896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:24.254278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea54c452a7f4bbcae1b52807543ee9d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.254951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea54c452a7f4bbcae1b52807543ee9d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.279154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '259508d5cb584fcda59e680e5222f1ff', 'request_id': 'req-c196c2b3-f70c-4d20-b779-728c749cce82'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.279578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea54c452a7f4bbcae1b52807543ee9d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.289368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '259508d5cb584fcda59e680e5222f1ff', 'request_id': 'req-c196c2b3-f70c-4d20-b779-728c749cce82'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.301100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea54c452a7f4bbcae1b52807543ee9d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.301237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '259508d5cb584fcda59e680e5222f1ff', 'request_id': 'req-c196c2b3-f70c-4d20-b779-728c749cce82'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.301832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e431f07c-3b22-48d8-ac06-9557797d88d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea54c452a7f4bbcae1b52807543ee9d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.313871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] The token cache is being invalidate because project 259508d5cb584fcda59e680e5222f1ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:24.314153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.314835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c196c2b3-f70c-4d20-b779-728c749cce82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:24.316679 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 935/1863] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:24 2022] DELETE /identity/v3/projects/259508d5cb584fcda59e680e5222f1ff => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:24.323190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5149af4-3235-4f99-96f8-96b203239e10 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.323500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5149af4-3235-4f99-96f8-96b203239e10 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.323733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5149af4-3235-4f99-96f8-96b203239e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.346110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 929/1864] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:24 2022] DELETE /identity/v3/users/0ea54c452a7f4bbcae1b52807543ee9d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:24.348176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d5149af4-3235-4f99-96f8-96b203239e10 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:24.349502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d5149af4-3235-4f99-96f8-96b203239e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:24.420674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:24.461233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c1615-7f8a-4059-b3fa-46ae135067d9 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.461558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c1615-7f8a-4059-b3fa-46ae135067d9 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.461857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c1615-7f8a-4059-b3fa-46ae135067d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.467887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 936/1865] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:24.484417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9b3c1615-7f8a-4059-b3fa-46ae135067d9 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:24.485869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9b3c1615-7f8a-4059-b3fa-46ae135067d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:24.499997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5149af4-3235-4f99-96f8-96b203239e10 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:24.506951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 930/1866] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:24.511544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5149af4-3235-4f99-96f8-96b203239e10 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:24.517277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769f12d-e602-407e-be13-edd0894657f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:24.517933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769f12d-e602-407e-be13-edd0894657f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.518211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769f12d-e602-407e-be13-edd0894657f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.518831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769f12d-e602-407e-be13-edd0894657f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.519559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769f12d-e602-407e-be13-edd0894657f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:24.519864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769f12d-e602-407e-be13-edd0894657f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:24.521637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769f12d-e602-407e-be13-edd0894657f1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:24.539433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 937/1867] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:22:24.541408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c1615-7f8a-4059-b3fa-46ae135067d9 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:24.553744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c1615-7f8a-4059-b3fa-46ae135067d9 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:24.561143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc404a-9d3e-4eef-8909-e5bd1ecc21b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:24.561804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc404a-9d3e-4eef-8909-e5bd1ecc21b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.562078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc404a-9d3e-4eef-8909-e5bd1ecc21b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.562346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc404a-9d3e-4eef-8909-e5bd1ecc21b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.562986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc404a-9d3e-4eef-8909-e5bd1ecc21b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:24.564985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc404a-9d3e-4eef-8909-e5bd1ecc21b6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:24.598327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 931/1868] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:24 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:24.602722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769f12d-e602-407e-be13-edd0894657f1 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:24.617326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769f12d-e602-407e-be13-edd0894657f1 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:24.729734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf34d1c-e083-4b53-b02a-09de32b53446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:24.730393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf34d1c-e083-4b53-b02a-09de32b53446 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.730682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf34d1c-e083-4b53-b02a-09de32b53446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.731024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf34d1c-e083-4b53-b02a-09de32b53446 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.732339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf34d1c-e083-4b53-b02a-09de32b53446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:24.734535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf34d1c-e083-4b53-b02a-09de32b53446 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:24.760099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 938/1869] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:24 2022] POST /identity/v3/users => generated 300 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:24.764604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc404a-9d3e-4eef-8909-e5bd1ecc21b6 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:24.776703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc404a-9d3e-4eef-8909-e5bd1ecc21b6 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:24.782333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9394ffb-531a-4703-834e-95f1777da3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:24.783742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9394ffb-531a-4703-834e-95f1777da3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.784082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9394ffb-531a-4703-834e-95f1777da3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.784756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9394ffb-531a-4703-834e-95f1777da3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.785449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9394ffb-531a-4703-834e-95f1777da3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:24.787509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9394ffb-531a-4703-834e-95f1777da3b8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:24.803090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 932/1870] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:24.805682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf34d1c-e083-4b53-b02a-09de32b53446 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:24.818060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf34d1c-e083-4b53-b02a-09de32b53446 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:24.823465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68222bbb-7ef1-430e-bdc2-e34a42e4c57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:24.824466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68222bbb-7ef1-430e-bdc2-e34a42e4c57a admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.824466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68222bbb-7ef1-430e-bdc2-e34a42e4c57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.824950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68222bbb-7ef1-430e-bdc2-e34a42e4c57a admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.825672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68222bbb-7ef1-430e-bdc2-e34a42e4c57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:24.827288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68222bbb-7ef1-430e-bdc2-e34a42e4c57a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:24.845601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 939/1871] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:24.846643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9394ffb-531a-4703-834e-95f1777da3b8 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:24.860081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9394ffb-531a-4703-834e-95f1777da3b8 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:24.876066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588826e-ce44-4a6e-9370-4331cfe90b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:24.877308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588826e-ce44-4a6e-9370-4331cfe90b6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.877308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588826e-ce44-4a6e-9370-4331cfe90b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.878340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588826e-ce44-4a6e-9370-4331cfe90b6c admin admin] PATH_INFO: `/v3/projects/aa1350b4600a40fdbeea316636fadd0a/users/ffa6ea74f1a94482a850db5d536a1e48/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:24.895483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588826e-ce44-4a6e-9370-4331cfe90b6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1350b4600a40fdbeea316636fadd0a, user_id=ffa6ea74f1a94482a850db5d536a1e48, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:24.898313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588826e-ce44-4a6e-9370-4331cfe90b6c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:24.929413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 933/1872] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:24 2022] PUT /identity/v3/projects/aa1350b4600a40fdbeea316636fadd0a/users/ffa6ea74f1a94482a850db5d536a1e48/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:24.945959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252813c-ede3-47b0-9d87-7220182e51a9 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:24.946298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252813c-ede3-47b0-9d87-7220182e51a9 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:24.946511 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252813c-ede3-47b0-9d87-7220182e51a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.025385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b252813c-ede3-47b0-9d87-7220182e51a9 None None] MFA Rules not processed for user `ffa6ea74f1a94482a850db5d536a1e48`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:25.027619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b252813c-ede3-47b0-9d87-7220182e51a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:25.148548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588826e-ce44-4a6e-9370-4331cfe90b6c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:25.155176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 940/1873] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:25.293710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588826e-ce44-4a6e-9370-4331cfe90b6c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:25.301128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cb4e3-c18e-4ac4-ab5b-583e8e43bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:25.302287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cb4e3-c18e-4ac4-ab5b-583e8e43bd64 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.302734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cb4e3-c18e-4ac4-ab5b-583e8e43bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.303349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cb4e3-c18e-4ac4-ab5b-583e8e43bd64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.304229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cb4e3-c18e-4ac4-ab5b-583e8e43bd64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:25.306687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cb4e3-c18e-4ac4-ab5b-583e8e43bd64 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:25.363021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 934/1874] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:25 2022] POST /identity/v3/projects => generated 401 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:25.367232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252813c-ede3-47b0-9d87-7220182e51a9 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:25.381846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252813c-ede3-47b0-9d87-7220182e51a9 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:25.430844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:25.492621 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347afd1a-7b2e-4422-9981-0c29e5d57b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:25.493710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347afd1a-7b2e-4422-9981-0c29e5d57b2e admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.494067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347afd1a-7b2e-4422-9981-0c29e5d57b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.494347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347afd1a-7b2e-4422-9981-0c29e5d57b2e admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.495147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347afd1a-7b2e-4422-9981-0c29e5d57b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:25.502418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347afd1a-7b2e-4422-9981-0c29e5d57b2e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:25.546281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 941/1875] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:25 2022] POST /identity/v3/users => generated 307 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:25.547484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cb4e3-c18e-4ac4-ab5b-583e8e43bd64 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:25.567004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347afd1a-7b2e-4422-9981-0c29e5d57b2e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:25.567300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cb4e3-c18e-4ac4-ab5b-583e8e43bd64 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:25.568902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-347afd1a-7b2e-4422-9981-0c29e5d57b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:25.577618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71430af0-73dd-4735-9c5a-29560ad642aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:25.578465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71430af0-73dd-4735-9c5a-29560ad642aa admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.579494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71430af0-73dd-4735-9c5a-29560ad642aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.579940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71430af0-73dd-4735-9c5a-29560ad642aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.581041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71430af0-73dd-4735-9c5a-29560ad642aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:25.583436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71430af0-73dd-4735-9c5a-29560ad642aa admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:25.592757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347afd1a-7b2e-4422-9981-0c29e5d57b2e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:25.613415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:25.614310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.616899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.617239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] PATH_INFO: `/v3/projects/94cf1e5c77d440edb7578b2420397055` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.618983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cf1e5c77d440edb7578b2420397055)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:25.623880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:25.633886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 935/1876] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:25.634824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71430af0-73dd-4735-9c5a-29560ad642aa admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:25.659649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71430af0-73dd-4735-9c5a-29560ad642aa admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:25.680839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8173e89-dabf-446e-9abb-76ad44be2cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:25.682572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8173e89-dabf-446e-9abb-76ad44be2cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.682883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8173e89-dabf-446e-9abb-76ad44be2cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.683634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8173e89-dabf-446e-9abb-76ad44be2cc3 admin admin] PATH_INFO: `/v3/projects/c81dbe50cdbe4dc9b20f6f47fb1db9b1/users/af8c2e6f07244c1d9899b699290d2558/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.720574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:25.723876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cf1e5c77d440edb7578b2420397055', 'request_id': 'req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:25.728874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8173e89-dabf-446e-9abb-76ad44be2cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81dbe50cdbe4dc9b20f6f47fb1db9b1, user_id=af8c2e6f07244c1d9899b699290d2558, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:25.733131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8173e89-dabf-446e-9abb-76ad44be2cc3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:25.742738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cf1e5c77d440edb7578b2420397055', 'request_id': 'req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:25.753432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 936/1877] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:25 2022] PUT /identity/v3/projects/c81dbe50cdbe4dc9b20f6f47fb1db9b1/users/af8c2e6f07244c1d9899b699290d2558/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:25.758824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8173e89-dabf-446e-9abb-76ad44be2cc3 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:25.759218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cf1e5c77d440edb7578b2420397055', 'request_id': 'req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:25.782203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] The token cache is being invalidate because project 94cf1e5c77d440edb7578b2420397055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:25.783218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:25.783218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-31d7d2ae-4ab8-4eec-976e-09fa8b8ab0f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:25.785409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 942/1878] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:25 2022] DELETE /identity/v3/projects/94cf1e5c77d440edb7578b2420397055 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:25.795297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2812b-883b-40fe-9555-318e05fd9d84 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.796000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2812b-883b-40fe-9555-318e05fd9d84 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.796697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2812b-883b-40fe-9555-318e05fd9d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.834945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f0e2812b-883b-40fe-9555-318e05fd9d84 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:25.837177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f0e2812b-883b-40fe-9555-318e05fd9d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:25.853797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8173e89-dabf-446e-9abb-76ad44be2cc3 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:25.861169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 943/1879] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:25.863664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4e325-b068-475a-87c6-e9f0cdc6215a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:25.864346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4e325-b068-475a-87c6-e9f0cdc6215a admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.864680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4e325-b068-475a-87c6-e9f0cdc6215a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.865018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4e325-b068-475a-87c6-e9f0cdc6215a admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.865755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4e325-b068-475a-87c6-e9f0cdc6215a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:25.868569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a361ae-fc20-49c9-a04d-56a33115574d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.868902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a361ae-fc20-49c9-a04d-56a33115574d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.869228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a361ae-fc20-49c9-a04d-56a33115574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.872296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4e325-b068-475a-87c6-e9f0cdc6215a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:25.891404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 937/1880] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:25.892423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4e325-b068-475a-87c6-e9f0cdc6215a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:25.894262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8aa4e325-b068-475a-87c6-e9f0cdc6215a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:25.913667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4e325-b068-475a-87c6-e9f0cdc6215a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:25.923130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf983616-9914-4adf-8856-6dd524a010b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:25.925211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf983616-9914-4adf-8856-6dd524a010b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.925803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf983616-9914-4adf-8856-6dd524a010b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.926267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf983616-9914-4adf-8856-6dd524a010b5 admin admin] PATH_INFO: `/v3/projects/c81dbe50cdbe4dc9b20f6f47fb1db9b1/users/af8c2e6f07244c1d9899b699290d2558/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:25.928827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf983616-9914-4adf-8856-6dd524a010b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81dbe50cdbe4dc9b20f6f47fb1db9b1, user_id=af8c2e6f07244c1d9899b699290d2558, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:25.930616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e2a361ae-fc20-49c9-a04d-56a33115574d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:25.932069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf983616-9914-4adf-8856-6dd524a010b5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:25.940039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e2a361ae-fc20-49c9-a04d-56a33115574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:25.948006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 938/1881] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:25 2022] PUT /identity/v3/projects/c81dbe50cdbe4dc9b20f6f47fb1db9b1/users/af8c2e6f07244c1d9899b699290d2558/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:25.960815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8702945-8ab0-4622-9609-f2b0bf9f35e3 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:25.961213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8702945-8ab0-4622-9609-f2b0bf9f35e3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:25.961646 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8702945-8ab0-4622-9609-f2b0bf9f35e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.006888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a8702945-8ab0-4622-9609-f2b0bf9f35e3 None None] MFA Rules not processed for user `af8c2e6f07244c1d9899b699290d2558`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:26.008239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a8702945-8ab0-4622-9609-f2b0bf9f35e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:26.071575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 944/1882] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:26.076970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a361ae-fc20-49c9-a04d-56a33115574d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.093627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a361ae-fc20-49c9-a04d-56a33115574d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.099643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9820da0-66ba-4f60-9a78-b92ce9d3ae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.100762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9820da0-66ba-4f60-9a78-b92ce9d3ae0c admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.101272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9820da0-66ba-4f60-9a78-b92ce9d3ae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.102290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9820da0-66ba-4f60-9a78-b92ce9d3ae0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.103731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9820da0-66ba-4f60-9a78-b92ce9d3ae0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:26.105695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9820da0-66ba-4f60-9a78-b92ce9d3ae0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:26.116514 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9820da0-66ba-4f60-9a78-b92ce9d3ae0c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:26.131382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 939/1883] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:25 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:26.147021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8702945-8ab0-4622-9609-f2b0bf9f35e3 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.147198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 945/1884] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:22:26.147310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a8702945-8ab0-4622-9609-f2b0bf9f35e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:26.150144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9820da0-66ba-4f60-9a78-b92ce9d3ae0c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.174893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9820da0-66ba-4f60-9a78-b92ce9d3ae0c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.185032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f0144-d738-4c2e-a7bd-6788f4b266ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.185906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f0144-d738-4c2e-a7bd-6788f4b266ec admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.187734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f0144-d738-4c2e-a7bd-6788f4b266ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.188156 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f0144-d738-4c2e-a7bd-6788f4b266ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.189706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f0144-d738-4c2e-a7bd-6788f4b266ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:26.192748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f0144-d738-4c2e-a7bd-6788f4b266ec admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:26.257751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 946/1885] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:26 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:26.258651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f0144-d738-4c2e-a7bd-6788f4b266ec admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.270595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:26.293048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8702945-8ab0-4622-9609-f2b0bf9f35e3 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.317647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:26.358918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f0144-d738-4c2e-a7bd-6788f4b266ec admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.367659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7aa4d4-124a-4d85-88ec-18239437f38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.368579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7aa4d4-124a-4d85-88ec-18239437f38c admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.368859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7aa4d4-124a-4d85-88ec-18239437f38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.369131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7aa4d4-124a-4d85-88ec-18239437f38c admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.370994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7aa4d4-124a-4d85-88ec-18239437f38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:26.373554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7aa4d4-124a-4d85-88ec-18239437f38c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:26.411958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 947/1886] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:26 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:26.413190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7aa4d4-124a-4d85-88ec-18239437f38c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.416037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fe7aa4d4-124a-4d85-88ec-18239437f38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:26.431342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3766-94f8-4c8f-9d16-fbc1c7e4314e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.431969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3766-94f8-4c8f-9d16-fbc1c7e4314e service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.432253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3766-94f8-4c8f-9d16-fbc1c7e4314e service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.432513 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3766-94f8-4c8f-9d16-fbc1c7e4314e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.445023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7aa4d4-124a-4d85-88ec-18239437f38c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.454195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.455261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.457416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.457934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] PATH_INFO: `/v3/users/0455304776e442e7b8c401dad99d9015` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.460519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0455304776e442e7b8c401dad99d9015)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:26.463538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:26.552079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed3766-94f8-4c8f-9d16-fbc1c7e4314e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:26.553579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed3766-94f8-4c8f-9d16-fbc1c7e4314e service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:26.561812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0455304776e442e7b8c401dad99d9015'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:26.562786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0455304776e442e7b8c401dad99d9015'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:26.575028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 940/1887] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:26 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:26.576492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3766-94f8-4c8f-9d16-fbc1c7e4314e service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.579465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0455304776e442e7b8c401dad99d9015'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:26.581094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0455304776e442e7b8c401dad99d9015'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:26.603481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3766-94f8-4c8f-9d16-fbc1c7e4314e service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.605181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8a649857-78f9-4058-9571-be0d575529c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0455304776e442e7b8c401dad99d9015'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:26.620824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676cb76-5e81-493e-b004-9638b89e8bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.621699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676cb76-5e81-493e-b004-9638b89e8bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.621795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676cb76-5e81-493e-b004-9638b89e8bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.622019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676cb76-5e81-493e-b004-9638b89e8bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.623289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676cb76-5e81-493e-b004-9638b89e8bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:26.624980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676cb76-5e81-493e-b004-9638b89e8bd9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:26.647111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 941/1888] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:26.648062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676cb76-5e81-493e-b004-9638b89e8bd9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.649951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e676cb76-5e81-493e-b004-9638b89e8bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:26.655376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 948/1889] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:26 2022] DELETE /identity/v3/users/0455304776e442e7b8c401dad99d9015 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:26.660100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97524f41-6a5c-4bcf-8d91-346cd3d022b1 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.660709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97524f41-6a5c-4bcf-8d91-346cd3d022b1 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.661232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97524f41-6a5c-4bcf-8d91-346cd3d022b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.665824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676cb76-5e81-493e-b004-9638b89e8bd9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.694192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-97524f41-6a5c-4bcf-8d91-346cd3d022b1 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:26.695114 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-97524f41-6a5c-4bcf-8d91-346cd3d022b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:26.765919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecebae1f-6fa4-44ea-9972-c45a663b024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.766602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebae1f-6fa4-44ea-9972-c45a663b024d admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.767420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebae1f-6fa4-44ea-9972-c45a663b024d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.767724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebae1f-6fa4-44ea-9972-c45a663b024d admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.768609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecebae1f-6fa4-44ea-9972-c45a663b024d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:26.770817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecebae1f-6fa4-44ea-9972-c45a663b024d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:26.787351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 942/1890] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:26.788494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecebae1f-6fa4-44ea-9972-c45a663b024d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.793616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ecebae1f-6fa4-44ea-9972-c45a663b024d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:26.793782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 949/1891] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:26.794710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97524f41-6a5c-4bcf-8d91-346cd3d022b1 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.806467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97524f41-6a5c-4bcf-8d91-346cd3d022b1 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.814028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bddb9-8585-4ca6-9752-a48b6984d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.814699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bddb9-8585-4ca6-9752-a48b6984d99f admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.814974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bddb9-8585-4ca6-9752-a48b6984d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.815241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bddb9-8585-4ca6-9752-a48b6984d99f admin admin] PATH_INFO: `/v3/projects/0d3f096c8c6c423680a789412cacb3db/users/d348b372bbce4b479305da5d77282e4d/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.824761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:26.833169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bddb9-8585-4ca6-9752-a48b6984d99f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3f096c8c6c423680a789412cacb3db, user_id=d348b372bbce4b479305da5d77282e4d, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:26.835593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bddb9-8585-4ca6-9752-a48b6984d99f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:26.862181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 950/1892] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:26 2022] PUT /identity/v3/projects/0d3f096c8c6c423680a789412cacb3db/users/d348b372bbce4b479305da5d77282e4d/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:26.863134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bddb9-8585-4ca6-9752-a48b6984d99f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:26.865988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8d3bddb9-8585-4ca6-9752-a48b6984d99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:26.875009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:26.879612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecebae1f-6fa4-44ea-9972-c45a663b024d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.968798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bddb9-8585-4ca6-9752-a48b6984d99f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:26.970738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c977b-eb7c-4f3a-8d2c-48ee6e653234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.971499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c977b-eb7c-4f3a-8d2c-48ee6e653234 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.971793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c977b-eb7c-4f3a-8d2c-48ee6e653234 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.972039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c977b-eb7c-4f3a-8d2c-48ee6e653234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.973083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1253-edba-4e74-9853-3508d3d0662f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:26.973753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1253-edba-4e74-9853-3508d3d0662f service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:26.974032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1253-edba-4e74-9853-3508d3d0662f service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:26.974310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1253-edba-4e74-9853-3508d3d0662f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:26.974693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8d4c977b-eb7c-4f3a-8d2c-48ee6e653234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:27.114635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c977b-eb7c-4f3a-8d2c-48ee6e653234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:27.116395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c977b-eb7c-4f3a-8d2c-48ee6e653234 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:27.130848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 943/1893] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:27.134540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd433ca5-665b-40ec-a7c3-71b039862752 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:27.135124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd433ca5-665b-40ec-a7c3-71b039862752 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:27.135747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd433ca5-665b-40ec-a7c3-71b039862752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:27.164608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f1253-edba-4e74-9853-3508d3d0662f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:27.166100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f1253-edba-4e74-9853-3508d3d0662f service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:27.175655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-bd433ca5-665b-40ec-a7c3-71b039862752 None None] MFA Rules not processed for user `d348b372bbce4b479305da5d77282e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:27.177678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-bd433ca5-665b-40ec-a7c3-71b039862752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:27.200691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 951/1894] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:27.314070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 944/1895] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:27.318971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1253-edba-4e74-9853-3508d3d0662f service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:27.334197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1253-edba-4e74-9853-3508d3d0662f service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:27.341881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ceee9-f006-4013-ab88-56c76909afbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:27.342949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ceee9-f006-4013-ab88-56c76909afbe admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:27.345596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ceee9-f006-4013-ab88-56c76909afbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:27.346173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ceee9-f006-4013-ab88-56c76909afbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:27.347553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ceee9-f006-4013-ab88-56c76909afbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:27.349980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ceee9-f006-4013-ab88-56c76909afbe admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:27.421352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 952/1896] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:27 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:22:27.426113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd433ca5-665b-40ec-a7c3-71b039862752 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:27.447299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd433ca5-665b-40ec-a7c3-71b039862752 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:27.597702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:27.623685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b97f5-0db1-4dd7-b361-3f2841fd6bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:27.624379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b97f5-0db1-4dd7-b361-3f2841fd6bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:27.624688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b97f5-0db1-4dd7-b361-3f2841fd6bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:27.624958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b97f5-0db1-4dd7-b361-3f2841fd6bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:27.625652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b97f5-0db1-4dd7-b361-3f2841fd6bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:27.631438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b97f5-0db1-4dd7-b361-3f2841fd6bd1 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:27.679864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 945/1897] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:27 2022] POST /identity/v3/users => generated 289 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:27.683321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ceee9-f006-4013-ab88-56c76909afbe admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:27.700170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ceee9-f006-4013-ab88-56c76909afbe admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:27.719948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3ab8d-38c1-4f04-852a-92d803b69137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:27.720582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3ab8d-38c1-4f04-852a-92d803b69137 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:27.720861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3ab8d-38c1-4f04-852a-92d803b69137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:27.721133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3ab8d-38c1-4f04-852a-92d803b69137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:27.732444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3ab8d-38c1-4f04-852a-92d803b69137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:27.738998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3ab8d-38c1-4f04-852a-92d803b69137 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:27.765749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 953/1898] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:27.772275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b97f5-0db1-4dd7-b361-3f2841fd6bd1 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:27.784182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b97f5-0db1-4dd7-b361-3f2841fd6bd1 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:27.793376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50598564-c983-4caf-a999-796d1830047b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:27.794905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50598564-c983-4caf-a999-796d1830047b admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:27.795527 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50598564-c983-4caf-a999-796d1830047b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:27.795697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50598564-c983-4caf-a999-796d1830047b admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:27.798099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50598564-c983-4caf-a999-796d1830047b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:27.800024 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50598564-c983-4caf-a999-796d1830047b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:27.814263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 946/1899] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:27.818380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3ab8d-38c1-4f04-852a-92d803b69137 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:27.832198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3ab8d-38c1-4f04-852a-92d803b69137 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:27.840142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a9fb0-03f8-4647-b635-20860204f32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:27.841080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a9fb0-03f8-4647-b635-20860204f32d admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:27.841360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a9fb0-03f8-4647-b635-20860204f32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:27.841707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a9fb0-03f8-4647-b635-20860204f32d admin admin] PATH_INFO: `/v3/projects/185667a964ce4cd692fb6209ffa65231/users/348a820bfd2240118d198e7722ebb466/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:27.869115 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a9fb0-03f8-4647-b635-20860204f32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=185667a964ce4cd692fb6209ffa65231, user_id=348a820bfd2240118d198e7722ebb466, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:27.880357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a9fb0-03f8-4647-b635-20860204f32d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:27.908485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 954/1900] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:27 2022] PUT /identity/v3/projects/185667a964ce4cd692fb6209ffa65231/users/348a820bfd2240118d198e7722ebb466/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:27.921715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df9efe-7e89-4ac9-9d42-6a408b3dbd0d None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:27.921994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df9efe-7e89-4ac9-9d42-6a408b3dbd0d None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:27.922331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df9efe-7e89-4ac9-9d42-6a408b3dbd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:27.958331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-64df9efe-7e89-4ac9-9d42-6a408b3dbd0d None None] MFA Rules not processed for user `348a820bfd2240118d198e7722ebb466`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:27.959559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-64df9efe-7e89-4ac9-9d42-6a408b3dbd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:28.112031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a9fb0-03f8-4647-b635-20860204f32d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:28.123938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 947/1901] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:28.181230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:28.205426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a9fb0-03f8-4647-b635-20860204f32d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:28.215142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed98eb6-f3bd-4837-a4f8-2c632dac9358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:28.216910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed98eb6-f3bd-4837-a4f8-2c632dac9358 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:28.217261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed98eb6-f3bd-4837-a4f8-2c632dac9358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:28.217645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed98eb6-f3bd-4837-a4f8-2c632dac9358 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:28.218403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed98eb6-f3bd-4837-a4f8-2c632dac9358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:28.220413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed98eb6-f3bd-4837-a4f8-2c632dac9358 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:28.260638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 955/1902] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:28 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:28.261997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df9efe-7e89-4ac9-9d42-6a408b3dbd0d None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:28.275132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df9efe-7e89-4ac9-9d42-6a408b3dbd0d None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:28.367825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409316a9-fc45-4aa3-9f17-eb2952b9a72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:28.368870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409316a9-fc45-4aa3-9f17-eb2952b9a72e admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:28.369613 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409316a9-fc45-4aa3-9f17-eb2952b9a72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:28.370138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409316a9-fc45-4aa3-9f17-eb2952b9a72e admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:28.371044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409316a9-fc45-4aa3-9f17-eb2952b9a72e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:28.373740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409316a9-fc45-4aa3-9f17-eb2952b9a72e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:28.404843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 948/1903] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:28 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:22:28.408633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed98eb6-f3bd-4837-a4f8-2c632dac9358 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:28.422976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409316a9-fc45-4aa3-9f17-eb2952b9a72e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:28.424233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed98eb6-f3bd-4837-a4f8-2c632dac9358 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:28.427148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-409316a9-fc45-4aa3-9f17-eb2952b9a72e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:28.440832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912c483-0309-48a8-92c6-f8e60c894377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:28.442137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912c483-0309-48a8-92c6-f8e60c894377 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:28.442876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912c483-0309-48a8-92c6-f8e60c894377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:28.443398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912c483-0309-48a8-92c6-f8e60c894377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:28.444309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912c483-0309-48a8-92c6-f8e60c894377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:28.446171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912c483-0309-48a8-92c6-f8e60c894377 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:28.461035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409316a9-fc45-4aa3-9f17-eb2952b9a72e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:28.475123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:28.477931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:28.478457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:28.478891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] PATH_INFO: `/v3/projects/552e1d71116e47f4a40689a642cfc8f6` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:28.479333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 956/1904] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:28.481985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912c483-0309-48a8-92c6-f8e60c894377 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:28.482551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=552e1d71116e47f4a40689a642cfc8f6)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:28.487555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:28.510775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912c483-0309-48a8-92c6-f8e60c894377 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:28.538054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5cd7c-5b50-4672-b6c7-b1626fc92254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:28.539944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5cd7c-5b50-4672-b6c7-b1626fc92254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:28.541877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5cd7c-5b50-4672-b6c7-b1626fc92254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:28.542752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5cd7c-5b50-4672-b6c7-b1626fc92254 admin admin] PATH_INFO: `/v3/projects/647cac807d814e379816ecd25e0ad18a/users/98d84cd466a04073a56d9228f1e80895/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:28.580577 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5cd7c-5b50-4672-b6c7-b1626fc92254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647cac807d814e379816ecd25e0ad18a, user_id=98d84cd466a04073a56d9228f1e80895, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:28.583379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5cd7c-5b50-4672-b6c7-b1626fc92254 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:28.607959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 957/1905] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:28 2022] PUT /identity/v3/projects/647cac807d814e379816ecd25e0ad18a/users/98d84cd466a04073a56d9228f1e80895/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:28.612714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5cd7c-5b50-4672-b6c7-b1626fc92254 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:28.638609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:28.666760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '552e1d71116e47f4a40689a642cfc8f6', 'request_id': 'req-439bf85c-44d5-4cb0-92f8-c55576b94ebd'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:28.684431 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '552e1d71116e47f4a40689a642cfc8f6', 'request_id': 'req-439bf85c-44d5-4cb0-92f8-c55576b94ebd'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:28.695389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '552e1d71116e47f4a40689a642cfc8f6', 'request_id': 'req-439bf85c-44d5-4cb0-92f8-c55576b94ebd'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:28.733178 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] The token cache is being invalidate because project 552e1d71116e47f4a40689a642cfc8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:28.733470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:28.736378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:28.737198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 949/1906] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:28 2022] DELETE /identity/v3/projects/552e1d71116e47f4a40689a642cfc8f6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:28.744942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:28.746393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:28.748423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5cd7c-5b50-4672-b6c7-b1626fc92254 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:28.841239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa7dba-d8ef-4589-a3bf-7789f81da44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:28.841644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa7dba-d8ef-4589-a3bf-7789f81da44d admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:28.842033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa7dba-d8ef-4589-a3bf-7789f81da44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:28.842683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa7dba-d8ef-4589-a3bf-7789f81da44d admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:28.843471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa7dba-d8ef-4589-a3bf-7789f81da44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:28.845386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa7dba-d8ef-4589-a3bf-7789f81da44d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:28.860853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 958/1907] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:28.889277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa7dba-d8ef-4589-a3bf-7789f81da44d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:28.894030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2efa7dba-d8ef-4589-a3bf-7789f81da44d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:28.917600 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa7dba-d8ef-4589-a3bf-7789f81da44d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:28.922977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c70160-4e2e-412d-a25f-64c5d4387c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:28.923643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c70160-4e2e-412d-a25f-64c5d4387c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:28.923917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c70160-4e2e-412d-a25f-64c5d4387c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:28.924188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c70160-4e2e-412d-a25f-64c5d4387c66 admin admin] PATH_INFO: `/v3/projects/647cac807d814e379816ecd25e0ad18a/users/98d84cd466a04073a56d9228f1e80895/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:28.926141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c70160-4e2e-412d-a25f-64c5d4387c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647cac807d814e379816ecd25e0ad18a, user_id=98d84cd466a04073a56d9228f1e80895, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:28.929173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c70160-4e2e-412d-a25f-64c5d4387c66 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:28.950028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 959/1908] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:28 2022] PUT /identity/v3/projects/647cac807d814e379816ecd25e0ad18a/users/98d84cd466a04073a56d9228f1e80895/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:28.954040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2d3cb-866e-4d23-94ae-63fb45328902 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:28.954040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2d3cb-866e-4d23-94ae-63fb45328902 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:28.954354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2d3cb-866e-4d23-94ae-63fb45328902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:28.989476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e9e2d3cb-866e-4d23-94ae-63fb45328902 None None] MFA Rules not processed for user `98d84cd466a04073a56d9228f1e80895`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:28.990741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e9e2d3cb-866e-4d23-94ae-63fb45328902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:28.996712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bf85c-44d5-4cb0-92f8-c55576b94ebd admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:29.003460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:29.140827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 960/1909] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:29.151687 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7c9c-69c5-412c-b3f7-7e39d28ccf94 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:29.152071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7c9c-69c5-412c-b3f7-7e39d28ccf94 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:29.152283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7c9c-69c5-412c-b3f7-7e39d28ccf94 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:29.154661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 961/1910] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:22:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:22:29.186098 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e20109-77c4-4e80-b009-5431fe08b2ff None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:29.186806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e20109-77c4-4e80-b009-5431fe08b2ff None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:29.187207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e20109-77c4-4e80-b009-5431fe08b2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:29.202104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:29.202759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:29.203141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:29.209770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] PATH_INFO: `/v3/users/1aeed3165b0549cdb71921c3f0c054f6` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:29.209770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aeed3165b0549cdb71921c3f0c054f6)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:29.209770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:29.238453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f2e20109-77c4-4e80-b009-5431fe08b2ff None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:29.243809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f2e20109-77c4-4e80-b009-5431fe08b2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:29.306071 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aeed3165b0549cdb71921c3f0c054f6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:29.307948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aeed3165b0549cdb71921c3f0c054f6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:29.333124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aeed3165b0549cdb71921c3f0c054f6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:29.343425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aeed3165b0549cdb71921c3f0c054f6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:29.353455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aeed3165b0549cdb71921c3f0c054f6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:29.374225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:29.386415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 950/1911] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:28 2022] DELETE /identity/v3/users/1aeed3165b0549cdb71921c3f0c054f6 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:29.389539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:29.399019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:29.410866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 962/1912] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:22:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:29.459370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e20109-77c4-4e80-b009-5431fe08b2ff None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:29.462742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f2e20109-77c4-4e80-b009-5431fe08b2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:29.514004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e20109-77c4-4e80-b009-5431fe08b2ff None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:29.524719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d88d0-37a5-4aba-bfc1-55d31334f4e7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:29.530635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:29.530887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:29.531156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:29.531500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:29.533404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:29.657020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dadf75-e978-455c-9e75-369c4d846ed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:29.658032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dadf75-e978-455c-9e75-369c4d846ed8 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:29.658319 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dadf75-e978-455c-9e75-369c4d846ed8 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:29.658756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dadf75-e978-455c-9e75-369c4d846ed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:29.668160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dadf75-e978-455c-9e75-369c4d846ed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:29.670389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dadf75-e978-455c-9e75-369c4d846ed8 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:29.670497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:29.671930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:29.683461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 951/1913] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:29.689714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 963/1914] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:29.738682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:29.740340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:29.751691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621f0d-3e65-40cb-93e2-003c30190028 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:29.755391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24f3ec-6256-4cb2-b2f2-227462c7a475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:29.756023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24f3ec-6256-4cb2-b2f2-227462c7a475 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:29.756297 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24f3ec-6256-4cb2-b2f2-227462c7a475 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:29.756568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24f3ec-6256-4cb2-b2f2-227462c7a475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:29.758483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9e24f3ec-6256-4cb2-b2f2-227462c7a475 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:29.850161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24f3ec-6256-4cb2-b2f2-227462c7a475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:29.850976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dadf75-e978-455c-9e75-369c4d846ed8 service nova] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:29.852758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-56dadf75-e978-455c-9e75-369c4d846ed8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:29.853099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24f3ec-6256-4cb2-b2f2-227462c7a475 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:29.870933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 952/1915] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:29.896730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:29.963838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dadf75-e978-455c-9e75-369c4d846ed8 service nova] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:29.985016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:29.985755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:29.986028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:29.986295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] PATH_INFO: `/v3/projects/9253fb977ae749d79b0a188dab556e0d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:29.987434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9253fb977ae749d79b0a188dab556e0d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:29.989395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:30.052130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9253fb977ae749d79b0a188dab556e0d', 'request_id': 'req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:30.061808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9253fb977ae749d79b0a188dab556e0d', 'request_id': 'req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:30.071850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9253fb977ae749d79b0a188dab556e0d', 'request_id': 'req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:30.090248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] The token cache is being invalidate because project 9253fb977ae749d79b0a188dab556e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:30.091748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:30.092794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c985b4c0-0afc-4147-b8c0-e11e0f8d1541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:30.101223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659769ac-ba68-4e6a-a801-582ba6ff45fd None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:30.101546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659769ac-ba68-4e6a-a801-582ba6ff45fd None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:30.101846 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659769ac-ba68-4e6a-a801-582ba6ff45fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:30.106130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 964/1916] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:29 2022] DELETE /identity/v3/projects/9253fb977ae749d79b0a188dab556e0d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:30.130726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-659769ac-ba68-4e6a-a801-582ba6ff45fd None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:30.131998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-659769ac-ba68-4e6a-a801-582ba6ff45fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:30.193426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:30.218033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 953/1917] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:30.227869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437db79-fdaa-41c6-b860-b27e0b47589a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:30.228241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437db79-fdaa-41c6-b860-b27e0b47589a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:30.228591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437db79-fdaa-41c6-b860-b27e0b47589a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:30.255233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8437db79-fdaa-41c6-b860-b27e0b47589a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:30.257667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8437db79-fdaa-41c6-b860-b27e0b47589a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:30.279248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 965/1918] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:30.283461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659769ac-ba68-4e6a-a801-582ba6ff45fd None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:30.297120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659769ac-ba68-4e6a-a801-582ba6ff45fd None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:30.303418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466bc61-ce33-40fc-bd19-bd1a564fd6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:30.304382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466bc61-ce33-40fc-bd19-bd1a564fd6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:30.304814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466bc61-ce33-40fc-bd19-bd1a564fd6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:30.305310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466bc61-ce33-40fc-bd19-bd1a564fd6e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:30.306454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466bc61-ce33-40fc-bd19-bd1a564fd6e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:30.306824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466bc61-ce33-40fc-bd19-bd1a564fd6e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:30.308888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466bc61-ce33-40fc-bd19-bd1a564fd6e5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:30.342449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 954/1919] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:22:30.350408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437db79-fdaa-41c6-b860-b27e0b47589a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:30.364416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437db79-fdaa-41c6-b860-b27e0b47589a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:30.381698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f2bd4-913c-4309-9d67-1f97d4969242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:30.382378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f2bd4-913c-4309-9d67-1f97d4969242 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:30.382710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f2bd4-913c-4309-9d67-1f97d4969242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:30.383084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f2bd4-913c-4309-9d67-1f97d4969242 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:30.383703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f2bd4-913c-4309-9d67-1f97d4969242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:30.390584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f2bd4-913c-4309-9d67-1f97d4969242 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:30.452956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 966/1920] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:30 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:30.457002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466bc61-ce33-40fc-bd19-bd1a564fd6e5 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:30.479619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466bc61-ce33-40fc-bd19-bd1a564fd6e5 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:30.610670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6cb6b-1701-4ce2-8c69-99f6e34b6dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:30.611489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6cb6b-1701-4ce2-8c69-99f6e34b6dad admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:30.612455 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6cb6b-1701-4ce2-8c69-99f6e34b6dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:30.612880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6cb6b-1701-4ce2-8c69-99f6e34b6dad admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:30.613756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6cb6b-1701-4ce2-8c69-99f6e34b6dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:30.616630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6cb6b-1701-4ce2-8c69-99f6e34b6dad admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:30.665796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 955/1921] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:30 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:30.671638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f2bd4-913c-4309-9d67-1f97d4969242 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:30.695724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f2bd4-913c-4309-9d67-1f97d4969242 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:30.720025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376341-f0f3-4854-85c2-6d697afb46dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:30.720835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376341-f0f3-4854-85c2-6d697afb46dc admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:30.721286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376341-f0f3-4854-85c2-6d697afb46dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:30.721724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61376341-f0f3-4854-85c2-6d697afb46dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:30.722521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61376341-f0f3-4854-85c2-6d697afb46dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:30.724255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61376341-f0f3-4854-85c2-6d697afb46dc admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:30.746299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 967/1922] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:30.748049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6cb6b-1701-4ce2-8c69-99f6e34b6dad admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:30.763939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6cb6b-1701-4ce2-8c69-99f6e34b6dad admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:30.777053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f4020-ee17-4779-9a23-804ee95587e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:30.777716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f4020-ee17-4779-9a23-804ee95587e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:30.778018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f4020-ee17-4779-9a23-804ee95587e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:30.778324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f4020-ee17-4779-9a23-804ee95587e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:30.779197 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f4020-ee17-4779-9a23-804ee95587e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:30.780884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f4020-ee17-4779-9a23-804ee95587e2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:30.807953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 956/1923] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:30.809463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376341-f0f3-4854-85c2-6d697afb46dc admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:30.823145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61376341-f0f3-4854-85c2-6d697afb46dc admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:30.829752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38fdd9-ba4f-4371-9d3f-ac76afe60026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:30.830393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38fdd9-ba4f-4371-9d3f-ac76afe60026 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:30.830802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38fdd9-ba4f-4371-9d3f-ac76afe60026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:30.831156 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38fdd9-ba4f-4371-9d3f-ac76afe60026 admin admin] PATH_INFO: `/v3/projects/3cfcac2b380940e99b568076438839b4/users/35153b38b81945fd900769a534210fff/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:30.847950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38fdd9-ba4f-4371-9d3f-ac76afe60026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cfcac2b380940e99b568076438839b4, user_id=35153b38b81945fd900769a534210fff, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:30.852122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38fdd9-ba4f-4371-9d3f-ac76afe60026 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:30.885649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 968/1924] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:30 2022] PUT /identity/v3/projects/3cfcac2b380940e99b568076438839b4/users/35153b38b81945fd900769a534210fff/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:30.897080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89158e47-10c9-45f7-a68e-4505bf7d3f1b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:30.897694 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89158e47-10c9-45f7-a68e-4505bf7d3f1b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:30.898500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89158e47-10c9-45f7-a68e-4505bf7d3f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:30.967457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-89158e47-10c9-45f7-a68e-4505bf7d3f1b None None] MFA Rules not processed for user `35153b38b81945fd900769a534210fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:30.968635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-89158e47-10c9-45f7-a68e-4505bf7d3f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:31.099063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 957/1925] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:31.107879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38fdd9-ba4f-4371-9d3f-ac76afe60026 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:31.114081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ad38fdd9-ba4f-4371-9d3f-ac76afe60026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:31.186900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:31.192055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38fdd9-ba4f-4371-9d3f-ac76afe60026 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:31.198385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d2e09-a028-4b58-bbd0-74797b702747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:31.199000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d2e09-a028-4b58-bbd0-74797b702747 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:31.199278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d2e09-a028-4b58-bbd0-74797b702747 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:31.199561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d2e09-a028-4b58-bbd0-74797b702747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:31.213715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d2e09-a028-4b58-bbd0-74797b702747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:31.216275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d2e09-a028-4b58-bbd0-74797b702747 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:31.228095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 969/1926] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:32.232770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89158e47-10c9-45f7-a68e-4505bf7d3f1b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:32.234768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-89158e47-10c9-45f7-a68e-4505bf7d3f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:32.300242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:32.362867 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89158e47-10c9-45f7-a68e-4505bf7d3f1b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:32.368651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:32.371098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:32.371098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:32.371098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] PATH_INFO: `/v3/users/ffa6ea74f1a94482a850db5d536a1e48` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:32.378139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa6ea74f1a94482a850db5d536a1e48)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:32.380336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:32.455645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa6ea74f1a94482a850db5d536a1e48'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:32.457780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa6ea74f1a94482a850db5d536a1e48'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:32.469707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa6ea74f1a94482a850db5d536a1e48'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:32.480049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa6ea74f1a94482a850db5d536a1e48'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:32.481086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa6ea74f1a94482a850db5d536a1e48'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:32.509342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 958/1927] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:32 2022] DELETE /identity/v3/users/ffa6ea74f1a94482a850db5d536a1e48 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:32.512201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154546-fe86-460e-8b09-64c0afc27fc2 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:32.513280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154546-fe86-460e-8b09-64c0afc27fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:32.513551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154546-fe86-460e-8b09-64c0afc27fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:32.538519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-cf154546-fe86-460e-8b09-64c0afc27fc2 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:32.540122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-cf154546-fe86-460e-8b09-64c0afc27fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:32.629561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 970/1928] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:32.638364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:32.639989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:32.749342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca109c-75af-43bb-a21c-64856b08e1db admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:32.753752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b917a-7a6b-432f-ab4e-3b146b68e454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:32.754740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b917a-7a6b-432f-ab4e-3b146b68e454 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:32.755026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b917a-7a6b-432f-ab4e-3b146b68e454 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:32.755302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b917a-7a6b-432f-ab4e-3b146b68e454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:32.788791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b917a-7a6b-432f-ab4e-3b146b68e454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:32.790255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b917a-7a6b-432f-ab4e-3b146b68e454 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:32.802673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 959/1929] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:33.026598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf154546-fe86-460e-8b09-64c0afc27fc2 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:33.029169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-cf154546-fe86-460e-8b09-64c0afc27fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:33.048273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf154546-fe86-460e-8b09-64c0afc27fc2 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:33.054969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:33.055878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:33.056381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:33.056758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] PATH_INFO: `/v3/users/d348b372bbce4b479305da5d77282e4d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:33.058654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d348b372bbce4b479305da5d77282e4d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:33.061629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:33.120538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd348b372bbce4b479305da5d77282e4d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.121742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd348b372bbce4b479305da5d77282e4d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.133376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd348b372bbce4b479305da5d77282e4d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.134585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd348b372bbce4b479305da5d77282e4d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.146554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd348b372bbce4b479305da5d77282e4d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.172451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 971/1930] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:33 2022] DELETE /identity/v3/users/d348b372bbce4b479305da5d77282e4d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:33.177039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5cee5-afd6-4d81-a1de-c9998469cd02 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:33.177395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5cee5-afd6-4d81-a1de-c9998469cd02 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:33.177769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5cee5-afd6-4d81-a1de-c9998469cd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:33.199811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-eda5cee5-afd6-4d81-a1de-c9998469cd02 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:33.202023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-eda5cee5-afd6-4d81-a1de-c9998469cd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:33.266975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:33.268438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:33.289985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 960/1931] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:33.301009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5cee5-afd6-4d81-a1de-c9998469cd02 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:33.303159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-eda5cee5-afd6-4d81-a1de-c9998469cd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:33.311026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:33.355030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114c3f0-f1a8-435c-8a37-0d47731e999d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:33.361711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:33.362522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:33.364022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:33.364596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] PATH_INFO: `/v3/projects/aa1350b4600a40fdbeea316636fadd0a` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:33.366785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1350b4600a40fdbeea316636fadd0a)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:33.369796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:33.372617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5cee5-afd6-4d81-a1de-c9998469cd02 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:33.377817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd761ae1-aaaa-4db7-8453-1e6175065c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:33.378517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd761ae1-aaaa-4db7-8453-1e6175065c75 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:33.378877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd761ae1-aaaa-4db7-8453-1e6175065c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:33.379218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd761ae1-aaaa-4db7-8453-1e6175065c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:33.381136 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:33.390500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd761ae1-aaaa-4db7-8453-1e6175065c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:33.391839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd761ae1-aaaa-4db7-8453-1e6175065c75 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:33.403036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 961/1932] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:33.429871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1350b4600a40fdbeea316636fadd0a', 'request_id': 'req-7cd6065d-a3ce-45a3-ac56-1990d2ded533'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.438253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1350b4600a40fdbeea316636fadd0a', 'request_id': 'req-7cd6065d-a3ce-45a3-ac56-1990d2ded533'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.445110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1350b4600a40fdbeea316636fadd0a', 'request_id': 'req-7cd6065d-a3ce-45a3-ac56-1990d2ded533'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.456789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] The token cache is being invalidate because project aa1350b4600a40fdbeea316636fadd0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:33.457149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.457919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.459428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 972/1933] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:33 2022] DELETE /identity/v3/projects/aa1350b4600a40fdbeea316636fadd0a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:33.463706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd761ae1-aaaa-4db7-8453-1e6175065c75 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:33.474990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd761ae1-aaaa-4db7-8453-1e6175065c75 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:33.536726 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:33.538879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:33.539157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:33.539426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] PATH_INFO: `/v3/users/af8c2e6f07244c1d9899b699290d2558` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:33.540811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af8c2e6f07244c1d9899b699290d2558)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:33.542777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:33.598404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af8c2e6f07244c1d9899b699290d2558'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.599237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af8c2e6f07244c1d9899b699290d2558'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.619322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af8c2e6f07244c1d9899b699290d2558'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.630078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af8c2e6f07244c1d9899b699290d2558'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.631414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af8c2e6f07244c1d9899b699290d2558'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.650703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 962/1934] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:33 2022] DELETE /identity/v3/users/af8c2e6f07244c1d9899b699290d2558 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:33.791759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:33.794411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:33.814034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6065d-a3ce-45a3-ac56-1990d2ded533 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:33.877143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:33.878070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:33.878422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:33.878760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] PATH_INFO: `/v3/projects/0d3f096c8c6c423680a789412cacb3db` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:33.880282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3f096c8c6c423680a789412cacb3db)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:33.882326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:33.940787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3f096c8c6c423680a789412cacb3db', 'request_id': 'req-9121537a-2d48-4063-a076-4c5de3d53e3d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.948819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3f096c8c6c423680a789412cacb3db', 'request_id': 'req-9121537a-2d48-4063-a076-4c5de3d53e3d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.956336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3f096c8c6c423680a789412cacb3db', 'request_id': 'req-9121537a-2d48-4063-a076-4c5de3d53e3d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.967135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] The token cache is being invalidate because project 0d3f096c8c6c423680a789412cacb3db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:33.967426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.968302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:33.973090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 973/1935] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:33 2022] DELETE /identity/v3/projects/0d3f096c8c6c423680a789412cacb3db => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:33.974644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:33.976117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:33.991535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e866bd-1525-4d11-9337-da134ba936e6 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.013538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:34.015684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:34.031374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:34.067633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:34.077411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.078376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.078654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.078946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] PATH_INFO: `/v3/users/348a820bfd2240118d198e7722ebb466` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.080104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348a820bfd2240118d198e7722ebb466)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.082249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.113588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121537a-2d48-4063-a076-4c5de3d53e3d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.119554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.120219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.120496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.120777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] PATH_INFO: `/v3/projects/c81dbe50cdbe4dc9b20f6f47fb1db9b1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.121961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c81dbe50cdbe4dc9b20f6f47fb1db9b1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.123865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.142290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348a820bfd2240118d198e7722ebb466'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.142974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348a820bfd2240118d198e7722ebb466'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.150710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348a820bfd2240118d198e7722ebb466'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.165383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348a820bfd2240118d198e7722ebb466'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.166236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5e066c6c-d90d-4c4e-868c-7e0d612209c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348a820bfd2240118d198e7722ebb466'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.178121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81dbe50cdbe4dc9b20f6f47fb1db9b1', 'request_id': 'req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.187208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81dbe50cdbe4dc9b20f6f47fb1db9b1', 'request_id': 'req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.188503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 963/1936] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:33 2022] DELETE /identity/v3/users/348a820bfd2240118d198e7722ebb466 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:34.198534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81dbe50cdbe4dc9b20f6f47fb1db9b1', 'request_id': 'req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.209808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] The token cache is being invalidate because project c81dbe50cdbe4dc9b20f6f47fb1db9b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:34.210100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.210773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6887d292-c4b0-4d7e-b7aa-4c7fd38bbe73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.212490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 974/1937] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:34 2022] DELETE /identity/v3/projects/c81dbe50cdbe4dc9b20f6f47fb1db9b1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:34.217701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816816e-47af-4a68-a794-8001dc84b8b9 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.217984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816816e-47af-4a68-a794-8001dc84b8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.218260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816816e-47af-4a68-a794-8001dc84b8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.239402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a816816e-47af-4a68-a794-8001dc84b8b9 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:34.240612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a816816e-47af-4a68-a794-8001dc84b8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:34.318819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 964/1938] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:34.327401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed94e0-1443-4b69-acc8-ea2a4224cf74 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.327401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed94e0-1443-4b69-acc8-ea2a4224cf74 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.327652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed94e0-1443-4b69-acc8-ea2a4224cf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.359912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-57ed94e0-1443-4b69-acc8-ea2a4224cf74 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:34.361954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-57ed94e0-1443-4b69-acc8-ea2a4224cf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:34.371176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 975/1939] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:34.375840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816816e-47af-4a68-a794-8001dc84b8b9 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:34.392909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816816e-47af-4a68-a794-8001dc84b8b9 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.399096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8ed7b-0447-45b7-bc95-2c6195c3ab6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.399717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8ed7b-0447-45b7-bc95-2c6195c3ab6a admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.399996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8ed7b-0447-45b7-bc95-2c6195c3ab6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.400278 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8ed7b-0447-45b7-bc95-2c6195c3ab6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.400977 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8ed7b-0447-45b7-bc95-2c6195c3ab6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:34.401843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8ed7b-0447-45b7-bc95-2c6195c3ab6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.403656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8ed7b-0447-45b7-bc95-2c6195c3ab6a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.417676 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 965/1940] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:22:34.421143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed94e0-1443-4b69-acc8-ea2a4224cf74 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:34.431724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed94e0-1443-4b69-acc8-ea2a4224cf74 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.437234 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9caca-61d6-42ef-9f33-ee5e17a1b08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.438012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9caca-61d6-42ef-9f33-ee5e17a1b08e admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.438395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9caca-61d6-42ef-9f33-ee5e17a1b08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.438760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9caca-61d6-42ef-9f33-ee5e17a1b08e admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.439501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9caca-61d6-42ef-9f33-ee5e17a1b08e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.442065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9caca-61d6-42ef-9f33-ee5e17a1b08e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.481865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 976/1941] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:34 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:34.483544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8ed7b-0447-45b7-bc95-2c6195c3ab6a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:34.494750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8ed7b-0447-45b7-bc95-2c6195c3ab6a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.533851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:34.581314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564d536-a991-468f-8a13-aacf81179790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.581969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564d536-a991-468f-8a13-aacf81179790 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.582245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564d536-a991-468f-8a13-aacf81179790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.582515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564d536-a991-468f-8a13-aacf81179790 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.583160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564d536-a991-468f-8a13-aacf81179790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.585387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9564d536-a991-468f-8a13-aacf81179790 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.618416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 966/1942] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:34 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:22:34.622805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9caca-61d6-42ef-9f33-ee5e17a1b08e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:34.635830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564d536-a991-468f-8a13-aacf81179790 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:34.638488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9564d536-a991-468f-8a13-aacf81179790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:34.640908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9caca-61d6-42ef-9f33-ee5e17a1b08e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.648080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bd113-aa44-49c8-aca5-b747dede6638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.649481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bd113-aa44-49c8-aca5-b747dede6638 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.650186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bd113-aa44-49c8-aca5-b747dede6638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.651990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bd113-aa44-49c8-aca5-b747dede6638 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.652905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564d536-a991-468f-8a13-aacf81179790 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.653083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bd113-aa44-49c8-aca5-b747dede6638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.655734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bd113-aa44-49c8-aca5-b747dede6638 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.659234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.660272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.660714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.661128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] PATH_INFO: `/v3/projects/185667a964ce4cd692fb6209ffa65231` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.662439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=185667a964ce4cd692fb6209ffa65231)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.664590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.669650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 977/1943] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:34.674300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bd113-aa44-49c8-aca5-b747dede6638 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:34.684767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bd113-aa44-49c8-aca5-b747dede6638 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.690789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb4521-59dd-496e-8465-f0296fdfeccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.692237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb4521-59dd-496e-8465-f0296fdfeccf admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.692588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb4521-59dd-496e-8465-f0296fdfeccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.692935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb4521-59dd-496e-8465-f0296fdfeccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.693694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb4521-59dd-496e-8465-f0296fdfeccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.696356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb4521-59dd-496e-8465-f0296fdfeccf admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.711628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 978/1944] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:34.712665 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb4521-59dd-496e-8465-f0296fdfeccf admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:34.724372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb4521-59dd-496e-8465-f0296fdfeccf admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.726831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185667a964ce4cd692fb6209ffa65231', 'request_id': 'req-619e9c04-caae-4303-9551-f444970143e6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.734947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185667a964ce4cd692fb6209ffa65231', 'request_id': 'req-619e9c04-caae-4303-9551-f444970143e6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.742847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185667a964ce4cd692fb6209ffa65231', 'request_id': 'req-619e9c04-caae-4303-9551-f444970143e6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.755733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] The token cache is being invalidate because project 185667a964ce4cd692fb6209ffa65231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:34.756022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.756703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:34.758646 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 967/1945] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:34 2022] DELETE /identity/v3/projects/185667a964ce4cd692fb6209ffa65231 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:34.765052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:34.765052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:34.800983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b83db-14f0-4dc8-9849-374851a3d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.801785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b83db-14f0-4dc8-9849-374851a3d2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.802067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b83db-14f0-4dc8-9849-374851a3d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.802347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b83db-14f0-4dc8-9849-374851a3d2ac admin admin] PATH_INFO: `/v3/projects/4d7bff2756f14dd0861b83c76efe6628/users/78eb58627d3d4597ac0581c0bc8c7969/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.814423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:34.824855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b83db-14f0-4dc8-9849-374851a3d2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d7bff2756f14dd0861b83c76efe6628, user_id=78eb58627d3d4597ac0581c0bc8c7969, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.828390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b83db-14f0-4dc8-9849-374851a3d2ac admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.843539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 979/1946] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:34 2022] PUT /identity/v3/projects/4d7bff2756f14dd0861b83c76efe6628/users/78eb58627d3d4597ac0581c0bc8c7969/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:34.853851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32f93-c9b9-487a-802f-14977c9dd02a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.854159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32f93-c9b9-487a-802f-14977c9dd02a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.854426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32f93-c9b9-487a-802f-14977c9dd02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.865045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e9c04-caae-4303-9551-f444970143e6 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:34.899658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-63e32f93-c9b9-487a-802f-14977c9dd02a None None] MFA Rules not processed for user `78eb58627d3d4597ac0581c0bc8c7969`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:34.900868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-63e32f93-c9b9-487a-802f-14977c9dd02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:34.924631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:34.942461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:34.943454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:34.943904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:34.944435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] PATH_INFO: `/v3/users/98d84cd466a04073a56d9228f1e80895` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:34.946862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98d84cd466a04073a56d9228f1e80895)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:34.949043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:34.999756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 980/1947] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:35.007469 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98d84cd466a04073a56d9228f1e80895'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.008145 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98d84cd466a04073a56d9228f1e80895'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.011086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32f93-c9b9-487a-802f-14977c9dd02a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.012562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-63e32f93-c9b9-487a-802f-14977c9dd02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:35.015673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98d84cd466a04073a56d9228f1e80895'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.024199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98d84cd466a04073a56d9228f1e80895'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.025188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98d84cd466a04073a56d9228f1e80895'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.044878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 968/1948] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:34 2022] DELETE /identity/v3/users/98d84cd466a04073a56d9228f1e80895 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:35.070344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32f93-c9b9-487a-802f-14977c9dd02a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.121855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.122517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.122790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.123063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.124810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:35.171071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:35.187303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.189132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.199492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 981/1949] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:35.223003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.224397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:35.291022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fab3b-4d75-42f6-9399-7b7d58b957b7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.295748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.296409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.296799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.297141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] PATH_INFO: `/v3/users/78eb58627d3d4597ac0581c0bc8c7969` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.298421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=78eb58627d3d4597ac0581c0bc8c7969)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.300522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.322101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.323601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:35.339366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2c932-0df4-4d35-bf44-253ba7ff1818 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.345303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.346015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.346293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.346564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] PATH_INFO: `/v3/projects/647cac807d814e379816ecd25e0ad18a` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.347697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=647cac807d814e379816ecd25e0ad18a)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.349811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.360197 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78eb58627d3d4597ac0581c0bc8c7969'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.361033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78eb58627d3d4597ac0581c0bc8c7969'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.368524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78eb58627d3d4597ac0581c0bc8c7969'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.376302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78eb58627d3d4597ac0581c0bc8c7969'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.377089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-017394fe-432c-46af-bed1-96ce840fce3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78eb58627d3d4597ac0581c0bc8c7969'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.384826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:35.398389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 969/1950] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:35 2022] DELETE /identity/v3/users/78eb58627d3d4597ac0581c0bc8c7969 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:35.398661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647cac807d814e379816ecd25e0ad18a', 'request_id': 'req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.402910 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282f51c-eed5-4d03-9531-efb541cc1aa9 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.403774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282f51c-eed5-4d03-9531-efb541cc1aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.404074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282f51c-eed5-4d03-9531-efb541cc1aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.407493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647cac807d814e379816ecd25e0ad18a', 'request_id': 'req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.414125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647cac807d814e379816ecd25e0ad18a', 'request_id': 'req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.423660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-5282f51c-eed5-4d03-9531-efb541cc1aa9 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:35.424839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5282f51c-eed5-4d03-9531-efb541cc1aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:35.425498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] The token cache is being invalidate because project 647cac807d814e379816ecd25e0ad18a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:35.425723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.426368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6d082e4f-d3ee-4b3d-ab7e-be76d46519d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:35.427917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 982/1951] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:35 2022] DELETE /identity/v3/projects/647cac807d814e379816ecd25e0ad18a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:35.434005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c35d6e-7ee0-4937-b731-bcfb268aba8f None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.434819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c35d6e-7ee0-4937-b731-bcfb268aba8f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.435154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c35d6e-7ee0-4937-b731-bcfb268aba8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.458693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-32c35d6e-7ee0-4937-b731-bcfb268aba8f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:35.459918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-32c35d6e-7ee0-4937-b731-bcfb268aba8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:35.488486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 970/1952] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:35.498936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282f51c-eed5-4d03-9531-efb541cc1aa9 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.500315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5282f51c-eed5-4d03-9531-efb541cc1aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:35.522758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 983/1953] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:35.531671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b1b04-b44b-4b1b-a28f-d44c32c19421 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.531954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b1b04-b44b-4b1b-a28f-d44c32c19421 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.532224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b1b04-b44b-4b1b-a28f-d44c32c19421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.552102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1f1b1b04-b44b-4b1b-a28f-d44c32c19421 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:35.553394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1f1b1b04-b44b-4b1b-a28f-d44c32c19421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:35.556961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282f51c-eed5-4d03-9531-efb541cc1aa9 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.561161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ec65b-749f-416d-bad4-8c4e412bb194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.561921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ec65b-749f-416d-bad4-8c4e412bb194 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.562074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 984/1954] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:35.562270 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ec65b-749f-416d-bad4-8c4e412bb194 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.562597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ec65b-749f-416d-bad4-8c4e412bb194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.566117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b1b04-b44b-4b1b-a28f-d44c32c19421 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.572791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ec65b-749f-416d-bad4-8c4e412bb194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.574174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ec65b-749f-416d-bad4-8c4e412bb194 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.580730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b1b04-b44b-4b1b-a28f-d44c32c19421 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.586460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e7bd3-5fd2-423f-99a9-787c20be4e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.587207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 971/1955] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:35.593434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e7bd3-5fd2-423f-99a9-787c20be4e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.593896 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e7bd3-5fd2-423f-99a9-787c20be4e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.594228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e7bd3-5fd2-423f-99a9-787c20be4e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.595687 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e7bd3-5fd2-423f-99a9-787c20be4e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:35.596088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e7bd3-5fd2-423f-99a9-787c20be4e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.597994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e7bd3-5fd2-423f-99a9-787c20be4e8b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.612601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 985/1956] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:22:35.619327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ec65b-749f-416d-bad4-8c4e412bb194 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.634473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ec65b-749f-416d-bad4-8c4e412bb194 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.641836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8740af-ddeb-458c-9fec-a936939b9ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.642554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8740af-ddeb-458c-9fec-a936939b9ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.643447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8740af-ddeb-458c-9fec-a936939b9ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.643906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8740af-ddeb-458c-9fec-a936939b9ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.644737 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8740af-ddeb-458c-9fec-a936939b9ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.647884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8740af-ddeb-458c-9fec-a936939b9ec9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.687633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 972/1957] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:35 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:35.692567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e7bd3-5fd2-423f-99a9-787c20be4e8b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.704345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e7bd3-5fd2-423f-99a9-787c20be4e8b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.763373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:35.776293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddc5d0-f9e2-44a5-b3b1-4c9666a059bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.777163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddc5d0-f9e2-44a5-b3b1-4c9666a059bb admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.777704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddc5d0-f9e2-44a5-b3b1-4c9666a059bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.778267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddc5d0-f9e2-44a5-b3b1-4c9666a059bb admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.778979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddc5d0-f9e2-44a5-b3b1-4c9666a059bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.781304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddc5d0-f9e2-44a5-b3b1-4c9666a059bb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.810617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 986/1958] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:35 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:35.811835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8740af-ddeb-458c-9fec-a936939b9ec9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.827577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8740af-ddeb-458c-9fec-a936939b9ec9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.849564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da0eec-3897-42a9-8424-86659b1bdbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.850672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da0eec-3897-42a9-8424-86659b1bdbd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.851083 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da0eec-3897-42a9-8424-86659b1bdbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.851366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da0eec-3897-42a9-8424-86659b1bdbd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.852788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da0eec-3897-42a9-8424-86659b1bdbd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.855173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da0eec-3897-42a9-8424-86659b1bdbd1 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.870674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 973/1959] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:35.877736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddc5d0-f9e2-44a5-b3b1-4c9666a059bb admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.890920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddc5d0-f9e2-44a5-b3b1-4c9666a059bb admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.896173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e1ee0-05df-4520-bf0b-29f5725360c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.896924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e1ee0-05df-4520-bf0b-29f5725360c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.897193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e1ee0-05df-4520-bf0b-29f5725360c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.897469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e1ee0-05df-4520-bf0b-29f5725360c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.898202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e1ee0-05df-4520-bf0b-29f5725360c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.899870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e1ee0-05df-4520-bf0b-29f5725360c9 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.916452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 987/1960] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:35.917747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da0eec-3897-42a9-8424-86659b1bdbd1 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:35.930329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da0eec-3897-42a9-8424-86659b1bdbd1 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:35.935324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287b696-62dc-4347-bd09-0371e5b6cf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:35.936179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287b696-62dc-4347-bd09-0371e5b6cf66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.936524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287b696-62dc-4347-bd09-0371e5b6cf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.936863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287b696-62dc-4347-bd09-0371e5b6cf66 admin admin] PATH_INFO: `/v3/projects/d461d0b4eaeb4a58af6e5a509b9ad652/users/6fbcfa5ad7d046f5832a9ed3fdf65fe2/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:35.954063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287b696-62dc-4347-bd09-0371e5b6cf66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d461d0b4eaeb4a58af6e5a509b9ad652, user_id=6fbcfa5ad7d046f5832a9ed3fdf65fe2, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:35.957493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287b696-62dc-4347-bd09-0371e5b6cf66 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:35.976579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 974/1961] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:35 2022] PUT /identity/v3/projects/d461d0b4eaeb4a58af6e5a509b9ad652/users/6fbcfa5ad7d046f5832a9ed3fdf65fe2/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:35.988425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923544e-5512-4716-a749-22725dd4d039 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:35.989489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923544e-5512-4716-a749-22725dd4d039 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:35.989489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923544e-5512-4716-a749-22725dd4d039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.032345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-0923544e-5512-4716-a749-22725dd4d039 None None] MFA Rules not processed for user `6fbcfa5ad7d046f5832a9ed3fdf65fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:36.034807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0923544e-5512-4716-a749-22725dd4d039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:36.131411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287b696-62dc-4347-bd09-0371e5b6cf66 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.133368 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2287b696-62dc-4347-bd09-0371e5b6cf66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:36.164044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 988/1962] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:36.173210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923544e-5512-4716-a749-22725dd4d039 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.204760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:36.255141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287b696-62dc-4347-bd09-0371e5b6cf66 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.256466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923544e-5512-4716-a749-22725dd4d039 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.260827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.261172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e5096-0610-4767-a922-5721b5b4ceee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.261633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.261802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e5096-0610-4767-a922-5721b5b4ceee admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.262042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.262134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e5096-0610-4767-a922-5721b5b4ceee admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.262349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e5096-0610-4767-a922-5721b5b4ceee admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.262518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] PATH_INFO: `/v3/projects/4d7bff2756f14dd0861b83c76efe6628` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.262980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e5096-0610-4767-a922-5721b5b4ceee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.263698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d7bff2756f14dd0861b83c76efe6628)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.264874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e5096-0610-4767-a922-5721b5b4ceee admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.265756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.292312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 989/1963] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:36 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:36.297081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e5096-0610-4767-a922-5721b5b4ceee admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.313925 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d7bff2756f14dd0861b83c76efe6628', 'request_id': 'req-00000f04-ed61-4f74-8261-2746a8f8eae3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:36.321761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d7bff2756f14dd0861b83c76efe6628', 'request_id': 'req-00000f04-ed61-4f74-8261-2746a8f8eae3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:36.328294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d7bff2756f14dd0861b83c76efe6628', 'request_id': 'req-00000f04-ed61-4f74-8261-2746a8f8eae3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:36.339130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] The token cache is being invalidate because project 4d7bff2756f14dd0861b83c76efe6628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:36.339352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:36.340064 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-00000f04-ed61-4f74-8261-2746a8f8eae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:36.342705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 975/1964] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:36 2022] DELETE /identity/v3/projects/4d7bff2756f14dd0861b83c76efe6628 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:36.346517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:36.347143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2b0b7-58ec-40c0-97b8-fd5552fbec47 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.347501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2b0b7-58ec-40c0-97b8-fd5552fbec47 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.347822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2b0b7-58ec-40c0-97b8-fd5552fbec47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.361362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e5096-0610-4767-a922-5721b5b4ceee admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.369222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f6a2b0b7-58ec-40c0-97b8-fd5552fbec47 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:36.370447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f6a2b0b7-58ec-40c0-97b8-fd5552fbec47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:36.419349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067c8d7-e686-430c-9248-a2db6d149182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.420032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067c8d7-e686-430c-9248-a2db6d149182 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.420221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067c8d7-e686-430c-9248-a2db6d149182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.420486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067c8d7-e686-430c-9248-a2db6d149182 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.421108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067c8d7-e686-430c-9248-a2db6d149182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.423071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067c8d7-e686-430c-9248-a2db6d149182 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.428956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 976/1965] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:36.438064 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca682699-aa56-40a5-8674-f8271d6548b4 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.438357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca682699-aa56-40a5-8674-f8271d6548b4 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.438663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca682699-aa56-40a5-8674-f8271d6548b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.447422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 990/1966] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:36 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:22:36.448659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067c8d7-e686-430c-9248-a2db6d149182 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.450422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7067c8d7-e686-430c-9248-a2db6d149182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:36.460865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ca682699-aa56-40a5-8674-f8271d6548b4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:36.462109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ca682699-aa56-40a5-8674-f8271d6548b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:36.463456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067c8d7-e686-430c-9248-a2db6d149182 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.469709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd068d-6aae-4448-b17f-9f5666333249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.470496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 977/1967] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:36.470972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd068d-6aae-4448-b17f-9f5666333249 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.471807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd068d-6aae-4448-b17f-9f5666333249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.472141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd068d-6aae-4448-b17f-9f5666333249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.472798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd068d-6aae-4448-b17f-9f5666333249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.474673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca682699-aa56-40a5-8674-f8271d6548b4 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.474781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd068d-6aae-4448-b17f-9f5666333249 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.484259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca682699-aa56-40a5-8674-f8271d6548b4 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.488808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 991/1968] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:36.488935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886dd8f-212b-4402-9df7-265f4cd65885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.489489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886dd8f-212b-4402-9df7-265f4cd65885 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.489815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886dd8f-212b-4402-9df7-265f4cd65885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.490077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd068d-6aae-4448-b17f-9f5666333249 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.490167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886dd8f-212b-4402-9df7-265f4cd65885 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.490715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886dd8f-212b-4402-9df7-265f4cd65885 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:36.491034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886dd8f-212b-4402-9df7-265f4cd65885 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.492610 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886dd8f-212b-4402-9df7-265f4cd65885 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.500250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd068d-6aae-4448-b17f-9f5666333249 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.504813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 978/1969] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:22:36.505254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca288aea-a9d2-4ee5-9311-9b65659f6f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.505934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca288aea-a9d2-4ee5-9311-9b65659f6f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.506213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca288aea-a9d2-4ee5-9311-9b65659f6f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.506487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca288aea-a9d2-4ee5-9311-9b65659f6f46 admin admin] PATH_INFO: `/v3/projects/5f9d03d5d6714c46ba4246445495b217/users/91a832606f724b779f77f62a8ca255eb/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.509051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886dd8f-212b-4402-9df7-265f4cd65885 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.519413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886dd8f-212b-4402-9df7-265f4cd65885 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.522132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca288aea-a9d2-4ee5-9311-9b65659f6f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9d03d5d6714c46ba4246445495b217, user_id=91a832606f724b779f77f62a8ca255eb, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.524008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b73b2-6866-4dbe-ac72-9306e048bf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.524597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b73b2-6866-4dbe-ac72-9306e048bf6c admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.524801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca288aea-a9d2-4ee5-9311-9b65659f6f46 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.524945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b73b2-6866-4dbe-ac72-9306e048bf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.525134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b73b2-6866-4dbe-ac72-9306e048bf6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.525792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b73b2-6866-4dbe-ac72-9306e048bf6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.527665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b73b2-6866-4dbe-ac72-9306e048bf6c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.539016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 992/1970] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:36 2022] PUT /identity/v3/projects/5f9d03d5d6714c46ba4246445495b217/users/91a832606f724b779f77f62a8ca255eb/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:36.542597 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca288aea-a9d2-4ee5-9311-9b65659f6f46 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.559410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 979/1971] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:36 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:22:36.563595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b73b2-6866-4dbe-ac72-9306e048bf6c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.591462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:36.606470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca288aea-a9d2-4ee5-9311-9b65659f6f46 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.629771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b73b2-6866-4dbe-ac72-9306e048bf6c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.634442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14daaa9-86f2-4cd0-9226-8862983425f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.635048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14daaa9-86f2-4cd0-9226-8862983425f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.635334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14daaa9-86f2-4cd0-9226-8862983425f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.635609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14daaa9-86f2-4cd0-9226-8862983425f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.636234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14daaa9-86f2-4cd0-9226-8862983425f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.638751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14daaa9-86f2-4cd0-9226-8862983425f6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.658060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:36.663880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 980/1972] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:36 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:36.665045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14daaa9-86f2-4cd0-9226-8862983425f6 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.668848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a11f0-edcb-442d-a52f-2e7cd6d2b5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.669466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a11f0-edcb-442d-a52f-2e7cd6d2b5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.669772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a11f0-edcb-442d-a52f-2e7cd6d2b5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.670045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a11f0-edcb-442d-a52f-2e7cd6d2b5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.670721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a11f0-edcb-442d-a52f-2e7cd6d2b5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.672359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a11f0-edcb-442d-a52f-2e7cd6d2b5c6 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.674682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14daaa9-86f2-4cd0-9226-8862983425f6 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.679193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561a267-d5ef-47bc-bc8b-718a033ea1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.679774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561a267-d5ef-47bc-bc8b-718a033ea1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.680050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561a267-d5ef-47bc-bc8b-718a033ea1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.680317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7561a267-d5ef-47bc-bc8b-718a033ea1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.680946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561a267-d5ef-47bc-bc8b-718a033ea1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.685927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 993/1973] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:36.686049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7561a267-d5ef-47bc-bc8b-718a033ea1c6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.686918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a11f0-edcb-442d-a52f-2e7cd6d2b5c6 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.693355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 981/1974] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:36.696931 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a11f0-edcb-442d-a52f-2e7cd6d2b5c6 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.697069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561a267-d5ef-47bc-bc8b-718a033ea1c6 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.701889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb5ce7-dfa3-4805-9ece-abb9ac24b581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.702530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb5ce7-dfa3-4805-9ece-abb9ac24b581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.702801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb5ce7-dfa3-4805-9ece-abb9ac24b581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.703069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb5ce7-dfa3-4805-9ece-abb9ac24b581 admin admin] PATH_INFO: `/v3/projects/5f9d03d5d6714c46ba4246445495b217/users/91a832606f724b779f77f62a8ca255eb/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.704944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb5ce7-dfa3-4805-9ece-abb9ac24b581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9d03d5d6714c46ba4246445495b217, user_id=91a832606f724b779f77f62a8ca255eb, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.707163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7561a267-d5ef-47bc-bc8b-718a033ea1c6 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.707574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb5ce7-dfa3-4805-9ece-abb9ac24b581 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.711769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa3301-6065-480b-b37f-d1b259fe50c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.712356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3301-6065-480b-b37f-d1b259fe50c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.712633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3301-6065-480b-b37f-d1b259fe50c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.712909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3301-6065-480b-b37f-d1b259fe50c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.713559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa3301-6065-480b-b37f-d1b259fe50c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.715312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa3301-6065-480b-b37f-d1b259fe50c0 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.718657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 994/1975] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:36 2022] PUT /identity/v3/projects/5f9d03d5d6714c46ba4246445495b217/users/91a832606f724b779f77f62a8ca255eb/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:36.726474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 982/1976] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:36.728711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c0687-8432-4a07-bb13-8de4dd2cbc3e None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.728995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c0687-8432-4a07-bb13-8de4dd2cbc3e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.729266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c0687-8432-4a07-bb13-8de4dd2cbc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.730449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa3301-6065-480b-b37f-d1b259fe50c0 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.740116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa3301-6065-480b-b37f-d1b259fe50c0 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.757182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-554c0687-8432-4a07-bb13-8de4dd2cbc3e None None] MFA Rules not processed for user `91a832606f724b779f77f62a8ca255eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:36.758401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-554c0687-8432-4a07-bb13-8de4dd2cbc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:36.800642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047e25d-4bf3-4649-9805-976d55798b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.801342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047e25d-4bf3-4649-9805-976d55798b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.801649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047e25d-4bf3-4649-9805-976d55798b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.801921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047e25d-4bf3-4649-9805-976d55798b02 admin admin] PATH_INFO: `/v3/projects/803d546865aa4d21bb03ae0be64dd005/users/8d66cb54736541d0818913b4b9b26569/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.817686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047e25d-4bf3-4649-9805-976d55798b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=803d546865aa4d21bb03ae0be64dd005, user_id=8d66cb54736541d0818913b4b9b26569, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:36.820383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047e25d-4bf3-4649-9805-976d55798b02 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:36.841398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 995/1977] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:36.842491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 983/1978] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:36 2022] PUT /identity/v3/projects/803d546865aa4d21bb03ae0be64dd005/users/8d66cb54736541d0818913b4b9b26569/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:36.852212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047e25d-4bf3-4649-9805-976d55798b02 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:36.852672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5266015-e99e-42fb-85af-c4a81a1cd437 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.853028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5266015-e99e-42fb-85af-c4a81a1cd437 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.853389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5266015-e99e-42fb-85af-c4a81a1cd437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.854077 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4047e25d-4bf3-4649-9805-976d55798b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:36.894572 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e5266015-e99e-42fb-85af-c4a81a1cd437 None None] MFA Rules not processed for user `8d66cb54736541d0818913b4b9b26569`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:36.896183 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e5266015-e99e-42fb-85af-c4a81a1cd437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:36.930806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047e25d-4bf3-4649-9805-976d55798b02 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:36.935553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6774e9-9a52-4213-a3ce-7539990b649d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:36.936182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6774e9-9a52-4213-a3ce-7539990b649d service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:36.936558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6774e9-9a52-4213-a3ce-7539990b649d service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:36.936832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6774e9-9a52-4213-a3ce-7539990b649d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:36.985410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:37.021225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6774e9-9a52-4213-a3ce-7539990b649d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:37.022709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6774e9-9a52-4213-a3ce-7539990b649d service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:37.024161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 996/1979] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:22:37.033593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 984/1980] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:37.034585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5266015-e99e-42fb-85af-c4a81a1cd437 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:37.036059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e5266015-e99e-42fb-85af-c4a81a1cd437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:37.048374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5266015-e99e-42fb-85af-c4a81a1cd437 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:37.052304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733125f-7a61-49b4-b254-0fe22b361c7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:37.053051 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733125f-7a61-49b4-b254-0fe22b361c7a service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:37.053331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733125f-7a61-49b4-b254-0fe22b361c7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:37.053691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733125f-7a61-49b4-b254-0fe22b361c7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:37.063454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733125f-7a61-49b4-b254-0fe22b361c7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:37.064864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733125f-7a61-49b4-b254-0fe22b361c7a service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:37.078881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 997/1981] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:37.710191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6774e9-9a52-4213-a3ce-7539990b649d service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:37.771189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6774e9-9a52-4213-a3ce-7539990b649d service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:37.775655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:37.776274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:37.776547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:37.776814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] PATH_INFO: `/v3/users/6fbcfa5ad7d046f5832a9ed3fdf65fe2` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:37.778439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fbcfa5ad7d046f5832a9ed3fdf65fe2)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:37.780333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:37.825182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fbcfa5ad7d046f5832a9ed3fdf65fe2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:37.825838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fbcfa5ad7d046f5832a9ed3fdf65fe2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:37.832644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fbcfa5ad7d046f5832a9ed3fdf65fe2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:37.839775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fbcfa5ad7d046f5832a9ed3fdf65fe2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:37.841124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fbcfa5ad7d046f5832a9ed3fdf65fe2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:37.858834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 985/1982] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:37 2022] DELETE /identity/v3/users/6fbcfa5ad7d046f5832a9ed3fdf65fe2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:37.863497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0525ad-31e6-40af-861d-e26c65452d5b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:37.863957 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0525ad-31e6-40af-861d-e26c65452d5b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:37.863957 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0525ad-31e6-40af-861d-e26c65452d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:37.882787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5d0525ad-31e6-40af-861d-e26c65452d5b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:37.883982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5d0525ad-31e6-40af-861d-e26c65452d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:37.916382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:37.944052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 998/1983] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:37.953621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:37.954985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:38.004152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:38.016550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743aef22-61ce-44b3-b358-8040ea3e4231 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:38.020497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2f30e-43bc-49f3-9bfd-e9d5f9dff82a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:38.021126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2f30e-43bc-49f3-9bfd-e9d5f9dff82a service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:38.021401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2f30e-43bc-49f3-9bfd-e9d5f9dff82a service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:38.021702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2f30e-43bc-49f3-9bfd-e9d5f9dff82a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:38.031602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2f30e-43bc-49f3-9bfd-e9d5f9dff82a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:38.033167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2f30e-43bc-49f3-9bfd-e9d5f9dff82a service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:38.043265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 986/1984] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:38.430457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0525ad-31e6-40af-861d-e26c65452d5b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:38.431893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5d0525ad-31e6-40af-861d-e26c65452d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:38.445827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0525ad-31e6-40af-861d-e26c65452d5b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:38.478792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:38.480745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:38.483137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:38.483812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] PATH_INFO: `/v3/projects/d461d0b4eaeb4a58af6e5a509b9ad652` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:38.487980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d461d0b4eaeb4a58af6e5a509b9ad652)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:38.490081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:38.558721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd461d0b4eaeb4a58af6e5a509b9ad652', 'request_id': 'req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.570719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd461d0b4eaeb4a58af6e5a509b9ad652', 'request_id': 'req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.577701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd461d0b4eaeb4a58af6e5a509b9ad652', 'request_id': 'req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.588036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] The token cache is being invalidate because project d461d0b4eaeb4a58af6e5a509b9ad652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:38.588319 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.588949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.591065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 999/1985] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:38 2022] DELETE /identity/v3/projects/d461d0b4eaeb4a58af6e5a509b9ad652 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:38.594659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2f30e-43bc-49f3-9bfd-e9d5f9dff82a service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:38.604315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2f30e-43bc-49f3-9bfd-e9d5f9dff82a service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:38.663225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:38.663864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:38.664135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:38.664399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] PATH_INFO: `/v3/users/91a832606f724b779f77f62a8ca255eb` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:38.665626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a832606f724b779f77f62a8ca255eb)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:38.667556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:38.716160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a832606f724b779f77f62a8ca255eb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.716767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a832606f724b779f77f62a8ca255eb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.723594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a832606f724b779f77f62a8ca255eb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.730606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a832606f724b779f77f62a8ca255eb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.731213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8dc29df9-97f5-4570-9d3e-74eee9ecf9da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a832606f724b779f77f62a8ca255eb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:38.748757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 987/1986] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:38 2022] DELETE /identity/v3/users/91a832606f724b779f77f62a8ca255eb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:39.016927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:39.019067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:39.035435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2940d3f-e0f1-40b6-b15e-2237b3c887f2 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:39.095256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:39.096037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.096315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.096589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] PATH_INFO: `/v3/projects/5f9d03d5d6714c46ba4246445495b217` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.097840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9d03d5d6714c46ba4246445495b217)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:39.099779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:39.149322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9d03d5d6714c46ba4246445495b217', 'request_id': 'req-da0a5807-afab-4bbe-a8f3-7d88768798b7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:39.158136 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9d03d5d6714c46ba4246445495b217', 'request_id': 'req-da0a5807-afab-4bbe-a8f3-7d88768798b7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:39.164469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9d03d5d6714c46ba4246445495b217', 'request_id': 'req-da0a5807-afab-4bbe-a8f3-7d88768798b7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:39.166569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:39.177798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] The token cache is being invalidate because project 5f9d03d5d6714c46ba4246445495b217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:39.178085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:39.178728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-da0a5807-afab-4bbe-a8f3-7d88768798b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:39.180311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1000/1987] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:39 2022] DELETE /identity/v3/projects/5f9d03d5d6714c46ba4246445495b217 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:39.185994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2e08-f416-4e28-aed0-6c10972aaec3 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.186594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2e08-f416-4e28-aed0-6c10972aaec3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.187014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a2e08-f416-4e28-aed0-6c10972aaec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.209399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-b34a2e08-f416-4e28-aed0-6c10972aaec3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:39.210705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b34a2e08-f416-4e28-aed0-6c10972aaec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:39.288556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 988/1988] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:39.298404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1a16-7369-4631-889f-20454cd75342 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.298971 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1a16-7369-4631-889f-20454cd75342 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.300554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1a16-7369-4631-889f-20454cd75342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.326483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6fff1a16-7369-4631-889f-20454cd75342 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:39.328632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6fff1a16-7369-4631-889f-20454cd75342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:39.353688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2e08-f416-4e28-aed0-6c10972aaec3 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:39.362272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1001/1989] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:39.371198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a2e08-f416-4e28-aed0-6c10972aaec3 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:39.379872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a908a-743c-48bc-947e-493357423509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:39.380961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a908a-743c-48bc-947e-493357423509 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.381426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a908a-743c-48bc-947e-493357423509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.381821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a908a-743c-48bc-947e-493357423509 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.382792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a908a-743c-48bc-947e-493357423509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:39.383273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a908a-743c-48bc-947e-493357423509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:39.385327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a908a-743c-48bc-947e-493357423509 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:39.388063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:39.407253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff1a16-7369-4631-889f-20454cd75342 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:39.412304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 989/1990] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:22:39.423915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff1a16-7369-4631-889f-20454cd75342 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:39.430756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ce24b-339f-4a64-be0d-0dca02258e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:39.431453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ce24b-339f-4a64-be0d-0dca02258e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.431797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ce24b-339f-4a64-be0d-0dca02258e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.432164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ce24b-339f-4a64-be0d-0dca02258e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.432865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ce24b-339f-4a64-be0d-0dca02258e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:39.435352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ce24b-339f-4a64-be0d-0dca02258e69 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:39.476374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1002/1991] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:39 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:22:39.486825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a908a-743c-48bc-947e-493357423509 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:39.496822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a908a-743c-48bc-947e-493357423509 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:39.561195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8739b-e62c-4f34-ab24-251375acaa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:39.561920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8739b-e62c-4f34-ab24-251375acaa37 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.562268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8739b-e62c-4f34-ab24-251375acaa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.562658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8739b-e62c-4f34-ab24-251375acaa37 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.563396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8739b-e62c-4f34-ab24-251375acaa37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:39.565658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8739b-e62c-4f34-ab24-251375acaa37 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:39.590386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 990/1992] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:39 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:22:39.595129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ce24b-339f-4a64-be0d-0dca02258e69 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:39.605638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ce24b-339f-4a64-be0d-0dca02258e69 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:39.610350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b386673-2a47-4f18-90c6-69dd6b187075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:39.610946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b386673-2a47-4f18-90c6-69dd6b187075 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.611225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b386673-2a47-4f18-90c6-69dd6b187075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.611495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b386673-2a47-4f18-90c6-69dd6b187075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.612175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b386673-2a47-4f18-90c6-69dd6b187075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:39.613814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b386673-2a47-4f18-90c6-69dd6b187075 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:39.624466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1003/1993] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:39.628446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8739b-e62c-4f34-ab24-251375acaa37 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:39.639244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8739b-e62c-4f34-ab24-251375acaa37 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:39.643853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd6149-4daa-4752-8fb2-abf8dc5f5502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:39.644515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd6149-4daa-4752-8fb2-abf8dc5f5502 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.644791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd6149-4daa-4752-8fb2-abf8dc5f5502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.645069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd6149-4daa-4752-8fb2-abf8dc5f5502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.645749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd6149-4daa-4752-8fb2-abf8dc5f5502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:39.647334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd6149-4daa-4752-8fb2-abf8dc5f5502 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:39.662148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 991/1994] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:39.663801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b386673-2a47-4f18-90c6-69dd6b187075 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:39.675878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b386673-2a47-4f18-90c6-69dd6b187075 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:39.680986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eae5e5-83c8-478f-a8b5-e0f638ed76c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:39.681879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eae5e5-83c8-478f-a8b5-e0f638ed76c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.682306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eae5e5-83c8-478f-a8b5-e0f638ed76c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.682723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eae5e5-83c8-478f-a8b5-e0f638ed76c4 admin admin] PATH_INFO: `/v3/projects/5a666c9d4266471088ffc6a45d2066fb/users/fe889b1bf81c48239653c9196abdf54e/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.704318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eae5e5-83c8-478f-a8b5-e0f638ed76c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a666c9d4266471088ffc6a45d2066fb, user_id=fe889b1bf81c48239653c9196abdf54e, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:39.707231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eae5e5-83c8-478f-a8b5-e0f638ed76c4 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:39.723249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1004/1995] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:39 2022] PUT /identity/v3/projects/5a666c9d4266471088ffc6a45d2066fb/users/fe889b1bf81c48239653c9196abdf54e/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:39.733892 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af9bf6-d024-4d95-acec-44fde415ba7e None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.734415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af9bf6-d024-4d95-acec-44fde415ba7e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.734415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af9bf6-d024-4d95-acec-44fde415ba7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.764131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-52af9bf6-d024-4d95-acec-44fde415ba7e None None] MFA Rules not processed for user `fe889b1bf81c48239653c9196abdf54e`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:39.765687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-52af9bf6-d024-4d95-acec-44fde415ba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:39.835968 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 992/1996] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:39.840398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eae5e5-83c8-478f-a8b5-e0f638ed76c4 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:39.901378 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eae5e5-83c8-478f-a8b5-e0f638ed76c4 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:39.906637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b29d-8c2e-4cfb-8827-466a52f2cc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:39.907234 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b29d-8c2e-4cfb-8827-466a52f2cc78 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:39.907504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b29d-8c2e-4cfb-8827-466a52f2cc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:39.907771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b29d-8c2e-4cfb-8827-466a52f2cc78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:39.908388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5b29d-8c2e-4cfb-8827-466a52f2cc78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:39.910447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5b29d-8c2e-4cfb-8827-466a52f2cc78 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:39.944133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1005/1997] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:39 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:39.945231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af9bf6-d024-4d95-acec-44fde415ba7e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:39.956578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af9bf6-d024-4d95-acec-44fde415ba7e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.010699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61a72a-2f47-480e-b882-f64669e978cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.011304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61a72a-2f47-480e-b882-f64669e978cf admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.011585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61a72a-2f47-480e-b882-f64669e978cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.011871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61a72a-2f47-480e-b882-f64669e978cf admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.012481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61a72a-2f47-480e-b882-f64669e978cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.015006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61a72a-2f47-480e-b882-f64669e978cf admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.023708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:40.036545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 993/1998] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:39 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:22:40.040536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b29d-8c2e-4cfb-8827-466a52f2cc78 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.050611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b29d-8c2e-4cfb-8827-466a52f2cc78 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.055165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764dbf8-d50d-4da4-9cf4-eeb66615b4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.055783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764dbf8-d50d-4da4-9cf4-eeb66615b4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.056060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764dbf8-d50d-4da4-9cf4-eeb66615b4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.056328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764dbf8-d50d-4da4-9cf4-eeb66615b4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.056965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764dbf8-d50d-4da4-9cf4-eeb66615b4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.058598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764dbf8-d50d-4da4-9cf4-eeb66615b4f8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.070575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1006/1999] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:40.074572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61a72a-2f47-480e-b882-f64669e978cf admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.084917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61a72a-2f47-480e-b882-f64669e978cf admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.089390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a5d078-8b6d-41c0-a612-5c62dac3e32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.090017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5d078-8b6d-41c0-a612-5c62dac3e32f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.090293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5d078-8b6d-41c0-a612-5c62dac3e32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.090778 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a5d078-8b6d-41c0-a612-5c62dac3e32f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.091227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5d078-8b6d-41c0-a612-5c62dac3e32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.092949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a5d078-8b6d-41c0-a612-5c62dac3e32f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.105767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 994/2000] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:40.106892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764dbf8-d50d-4da4-9cf4-eeb66615b4f8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.116714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764dbf8-d50d-4da4-9cf4-eeb66615b4f8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.121354 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc77e-94f5-425e-b2b5-71aaff7b1441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.122040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc77e-94f5-425e-b2b5-71aaff7b1441 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.122839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc77e-94f5-425e-b2b5-71aaff7b1441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.123102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc77e-94f5-425e-b2b5-71aaff7b1441 admin admin] PATH_INFO: `/v3/projects/2cb4082572474d1c9635458745d57b01/users/eabf7aa514ad462e8b155bf19b5cf6d7/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.137168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddc77e-94f5-425e-b2b5-71aaff7b1441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb4082572474d1c9635458745d57b01, user_id=eabf7aa514ad462e8b155bf19b5cf6d7, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.140479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddc77e-94f5-425e-b2b5-71aaff7b1441 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.153416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1007/2001] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:40 2022] PUT /identity/v3/projects/2cb4082572474d1c9635458745d57b01/users/eabf7aa514ad462e8b155bf19b5cf6d7/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:40.164338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf9fd8-a667-4719-b44f-5e9a6b912e59 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.164628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf9fd8-a667-4719-b44f-5e9a6b912e59 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.164903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf9fd8-a667-4719-b44f-5e9a6b912e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.192937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-90cf9fd8-a667-4719-b44f-5e9a6b912e59 None None] MFA Rules not processed for user `eabf7aa514ad462e8b155bf19b5cf6d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:40.194176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-90cf9fd8-a667-4719-b44f-5e9a6b912e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:40.256412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 995/2002] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:40.260941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc77e-94f5-425e-b2b5-71aaff7b1441 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.301702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:40.323669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc77e-94f5-425e-b2b5-71aaff7b1441 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.328263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d0743-9751-48a5-8e8b-bd0c62589eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.328867 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d0743-9751-48a5-8e8b-bd0c62589eba admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.329140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d0743-9751-48a5-8e8b-bd0c62589eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.329408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d0743-9751-48a5-8e8b-bd0c62589eba admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.330057 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d0743-9751-48a5-8e8b-bd0c62589eba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.331947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d0743-9751-48a5-8e8b-bd0c62589eba admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.358146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1008/2003] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:40 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:40.362297 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf9fd8-a667-4719-b44f-5e9a6b912e59 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.374019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf9fd8-a667-4719-b44f-5e9a6b912e59 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.429354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2748f01-4afa-409e-815f-b3397d53c79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.430049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2748f01-4afa-409e-815f-b3397d53c79f admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.430332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2748f01-4afa-409e-815f-b3397d53c79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.430634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2748f01-4afa-409e-815f-b3397d53c79f admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.431220 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2748f01-4afa-409e-815f-b3397d53c79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.433273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2748f01-4afa-409e-815f-b3397d53c79f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.455731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 996/2004] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:40 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:40.459884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d0743-9751-48a5-8e8b-bd0c62589eba admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.470488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d0743-9751-48a5-8e8b-bd0c62589eba admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.475413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef002-8f8b-4f67-8676-2b58b3bf053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.476024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef002-8f8b-4f67-8676-2b58b3bf053c admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.476309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef002-8f8b-4f67-8676-2b58b3bf053c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.476606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef002-8f8b-4f67-8676-2b58b3bf053c admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.477339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef002-8f8b-4f67-8676-2b58b3bf053c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.479003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef002-8f8b-4f67-8676-2b58b3bf053c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.494304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1009/2005] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:22:40.495928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2748f01-4afa-409e-815f-b3397d53c79f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.507439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2748f01-4afa-409e-815f-b3397d53c79f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.514592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a71e1-48b1-4073-8918-e6a6ff900068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.515403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a71e1-48b1-4073-8918-e6a6ff900068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.515760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a71e1-48b1-4073-8918-e6a6ff900068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.516111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a71e1-48b1-4073-8918-e6a6ff900068 admin admin] PATH_INFO: `/v3/projects/a9cc5feeb3cb4edcbbff127a5b6a81af/users/4611e05ead4b407c8b2367d2ed76d69d/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.532839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a71e1-48b1-4073-8918-e6a6ff900068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9cc5feeb3cb4edcbbff127a5b6a81af, user_id=4611e05ead4b407c8b2367d2ed76d69d, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.535832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a71e1-48b1-4073-8918-e6a6ff900068 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.555753 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 997/2006] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:40 2022] PUT /identity/v3/projects/a9cc5feeb3cb4edcbbff127a5b6a81af/users/4611e05ead4b407c8b2367d2ed76d69d/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:40.560562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef002-8f8b-4f67-8676-2b58b3bf053c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.576024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef002-8f8b-4f67-8676-2b58b3bf053c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.678318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b90195-218e-42ff-9cb9-2e9523431ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.678968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b90195-218e-42ff-9cb9-2e9523431ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.688890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b90195-218e-42ff-9cb9-2e9523431ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.688890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b90195-218e-42ff-9cb9-2e9523431ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.688890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b90195-218e-42ff-9cb9-2e9523431ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.688890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b90195-218e-42ff-9cb9-2e9523431ecc admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.695482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1010/2007] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:40.699620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a71e1-48b1-4073-8918-e6a6ff900068 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.713660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a71e1-48b1-4073-8918-e6a6ff900068 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.719006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00b556-64ce-442e-9eea-4049c704d9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.719887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00b556-64ce-442e-9eea-4049c704d9d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.720234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00b556-64ce-442e-9eea-4049c704d9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.720603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00b556-64ce-442e-9eea-4049c704d9d6 admin admin] PATH_INFO: `/v3/projects/a9cc5feeb3cb4edcbbff127a5b6a81af/users/4611e05ead4b407c8b2367d2ed76d69d/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.722754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00b556-64ce-442e-9eea-4049c704d9d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9cc5feeb3cb4edcbbff127a5b6a81af, user_id=4611e05ead4b407c8b2367d2ed76d69d, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:40.726062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d00b556-64ce-442e-9eea-4049c704d9d6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:40.737842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 998/2008] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:40 2022] PUT /identity/v3/projects/a9cc5feeb3cb4edcbbff127a5b6a81af/users/4611e05ead4b407c8b2367d2ed76d69d/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:40.747756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b816cc-863c-4414-a662-91948a425802 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.748036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b816cc-863c-4414-a662-91948a425802 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.748306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b816cc-863c-4414-a662-91948a425802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:40.776873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-78b816cc-863c-4414-a662-91948a425802 None None] MFA Rules not processed for user `4611e05ead4b407c8b2367d2ed76d69d`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:40.778141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-78b816cc-863c-4414-a662-91948a425802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:40.849018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1011/2009] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:40.861655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00b556-64ce-442e-9eea-4049c704d9d6 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:40.863664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1d00b556-64ce-442e-9eea-4049c704d9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:40.906508 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:40.941708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00b556-64ce-442e-9eea-4049c704d9d6 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:40.945538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db849f1-35d5-4606-ad30-7fd551536263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:40.946152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db849f1-35d5-4606-ad30-7fd551536263 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:40.946426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db849f1-35d5-4606-ad30-7fd551536263 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:40.946698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db849f1-35d5-4606-ad30-7fd551536263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:41.003336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db849f1-35d5-4606-ad30-7fd551536263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:41.004761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db849f1-35d5-4606-ad30-7fd551536263 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:41.014045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 999/2010] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:22:45.692029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b816cc-863c-4414-a662-91948a425802 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:45.704073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b816cc-863c-4414-a662-91948a425802 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:45.708048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60acb97-302b-47d7-9935-116fde37c333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:45.708656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60acb97-302b-47d7-9935-116fde37c333 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:45.708922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60acb97-302b-47d7-9935-116fde37c333 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:45.709187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60acb97-302b-47d7-9935-116fde37c333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:45.760874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:45.804727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60acb97-302b-47d7-9935-116fde37c333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:45.811538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60acb97-302b-47d7-9935-116fde37c333 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:45.822116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1012/2011] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:46.707573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db849f1-35d5-4606-ad30-7fd551536263 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:46.716067 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db849f1-35d5-4606-ad30-7fd551536263 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:46.719707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262126f-9d58-49ad-88be-2201eb0c7aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:46.720299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262126f-9d58-49ad-88be-2201eb0c7aab service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:46.720574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262126f-9d58-49ad-88be-2201eb0c7aab service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:46.720840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262126f-9d58-49ad-88be-2201eb0c7aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:46.730390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262126f-9d58-49ad-88be-2201eb0c7aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:46.731894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262126f-9d58-49ad-88be-2201eb0c7aab service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:46.741335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1000/2012] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:55.585662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60acb97-302b-47d7-9935-116fde37c333 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:55.587504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f60acb97-302b-47d7-9935-116fde37c333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:55.666915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60acb97-302b-47d7-9935-116fde37c333 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:55.673043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:55.673719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:55.674007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:55.674280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] PATH_INFO: `/v3/users/8d66cb54736541d0818913b4b9b26569` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:55.675607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d66cb54736541d0818913b4b9b26569)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:55.677677 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:55.763284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d66cb54736541d0818913b4b9b26569'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:55.765665 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d66cb54736541d0818913b4b9b26569'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:55.774847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d66cb54736541d0818913b4b9b26569'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:55.775500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d66cb54736541d0818913b4b9b26569'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:55.787261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d66cb54736541d0818913b4b9b26569'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:55.813711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1013/2013] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:55 2022] DELETE /identity/v3/users/8d66cb54736541d0818913b4b9b26569 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:55.818470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538624f0-5038-4e7b-903c-ce875af5f90f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:55.819037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538624f0-5038-4e7b-903c-ce875af5f90f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:55.819037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538624f0-5038-4e7b-903c-ce875af5f90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:55.876349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-538624f0-5038-4e7b-903c-ce875af5f90f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:55.878967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-538624f0-5038-4e7b-903c-ce875af5f90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:55.974466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1001/2014] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:55.987518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:55.996611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:56.112523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fea0ae-fffb-48e7-ac0b-4a8ea49e95cb admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:56.123832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ab2d4-6e3c-45f3-8538-52de75f33b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:56.124711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab2d4-6e3c-45f3-8538-52de75f33b45 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:56.125156 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab2d4-6e3c-45f3-8538-52de75f33b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:56.125632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ab2d4-6e3c-45f3-8538-52de75f33b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:56.141495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:56.148670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ab2d4-6e3c-45f3-8538-52de75f33b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:56.152499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ab2d4-6e3c-45f3-8538-52de75f33b45 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:56.164096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1014/2015] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:56.471225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538624f0-5038-4e7b-903c-ce875af5f90f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:56.472731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-538624f0-5038-4e7b-903c-ce875af5f90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:56.485625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538624f0-5038-4e7b-903c-ce875af5f90f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:56.490477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:56.491204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:56.491553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:56.491918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] PATH_INFO: `/v3/projects/803d546865aa4d21bb03ae0be64dd005` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:56.493106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=803d546865aa4d21bb03ae0be64dd005)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:56.495645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:56.557810 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803d546865aa4d21bb03ae0be64dd005', 'request_id': 'req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:56.568253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803d546865aa4d21bb03ae0be64dd005', 'request_id': 'req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:56.571420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:56.578626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803d546865aa4d21bb03ae0be64dd005', 'request_id': 'req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:56.593800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] The token cache is being invalidate because project 803d546865aa4d21bb03ae0be64dd005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:56.594751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:56.595542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ebe1cd6f-834f-463b-8e8d-0c26110bce0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:56.597418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1002/2016] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:56 2022] DELETE /identity/v3/projects/803d546865aa4d21bb03ae0be64dd005 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:56.602564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9660d4-a443-456d-bb0f-052d7f525abe None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:56.602564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9660d4-a443-456d-bb0f-052d7f525abe None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:56.603100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9660d4-a443-456d-bb0f-052d7f525abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:56.661611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ed9660d4-a443-456d-bb0f-052d7f525abe None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:56.662898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ed9660d4-a443-456d-bb0f-052d7f525abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:56.772409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1015/2017] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:56.785136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092425e4-5378-4431-98c0-ac154fa31594 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:56.785826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092425e4-5378-4431-98c0-ac154fa31594 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:56.785954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092425e4-5378-4431-98c0-ac154fa31594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:56.815787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-092425e4-5378-4431-98c0-ac154fa31594 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:56.817536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-092425e4-5378-4431-98c0-ac154fa31594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:56.826564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1003/2018] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:56.831894 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9660d4-a443-456d-bb0f-052d7f525abe None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:56.843824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9660d4-a443-456d-bb0f-052d7f525abe None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:56.850672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba6ab7-9882-4e0a-94a3-2ed4ed5104c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:56.850672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba6ab7-9882-4e0a-94a3-2ed4ed5104c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:56.853648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba6ab7-9882-4e0a-94a3-2ed4ed5104c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:56.854194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba6ab7-9882-4e0a-94a3-2ed4ed5104c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:56.857280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba6ab7-9882-4e0a-94a3-2ed4ed5104c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:22:56.858833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba6ab7-9882-4e0a-94a3-2ed4ed5104c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:56.869617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba6ab7-9882-4e0a-94a3-2ed4ed5104c8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:56.927587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1016/2019] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:22:56.937663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092425e4-5378-4431-98c0-ac154fa31594 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:56.972528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092425e4-5378-4431-98c0-ac154fa31594 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:56.986304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935e4ec-0a84-4913-943d-e537c24fb3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:56.993833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935e4ec-0a84-4913-943d-e537c24fb3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:56.994366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935e4ec-0a84-4913-943d-e537c24fb3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:56.994709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935e4ec-0a84-4913-943d-e537c24fb3c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:57.001444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935e4ec-0a84-4913-943d-e537c24fb3c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:57.003480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935e4ec-0a84-4913-943d-e537c24fb3c4 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:57.067372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1004/2020] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:22:56 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:57.072214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba6ab7-9882-4e0a-94a3-2ed4ed5104c8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:57.086532 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba6ab7-9882-4e0a-94a3-2ed4ed5104c8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:57.188341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa44b3-280a-49e2-ab19-b21508a63d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:57.188962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa44b3-280a-49e2-ab19-b21508a63d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:57.189239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa44b3-280a-49e2-ab19-b21508a63d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:57.189506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa44b3-280a-49e2-ab19-b21508a63d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:57.190300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa44b3-280a-49e2-ab19-b21508a63d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:57.192790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa44b3-280a-49e2-ab19-b21508a63d9b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:57.230948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1017/2021] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:22:57 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:22:57.235412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935e4ec-0a84-4913-943d-e537c24fb3c4 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:57.252884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935e4ec-0a84-4913-943d-e537c24fb3c4 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:57.264943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8661c-0f15-4dca-8d43-93718953a2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:57.265705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8661c-0f15-4dca-8d43-93718953a2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:57.266310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8661c-0f15-4dca-8d43-93718953a2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:57.266741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8661c-0f15-4dca-8d43-93718953a2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:57.267639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8661c-0f15-4dca-8d43-93718953a2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:57.269807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8661c-0f15-4dca-8d43-93718953a2d9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:57.299702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1005/2022] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:57.305960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa44b3-280a-49e2-ab19-b21508a63d9b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:57.327620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa44b3-280a-49e2-ab19-b21508a63d9b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:57.343203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57e2cb-1417-4904-9d2c-5e5f1b54f993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:57.345197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e2cb-1417-4904-9d2c-5e5f1b54f993 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:57.346286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e2cb-1417-4904-9d2c-5e5f1b54f993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:57.347491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57e2cb-1417-4904-9d2c-5e5f1b54f993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:57.348553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57e2cb-1417-4904-9d2c-5e5f1b54f993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:57.350778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57e2cb-1417-4904-9d2c-5e5f1b54f993 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:57.368880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1018/2023] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:22:57.373896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8661c-0f15-4dca-8d43-93718953a2d9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:57.390670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8661c-0f15-4dca-8d43-93718953a2d9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:57.398965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e72249-1c41-4d27-bbb3-5ebc609c717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:57.399817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e72249-1c41-4d27-bbb3-5ebc609c717e admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:57.400353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e72249-1c41-4d27-bbb3-5ebc609c717e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:57.400832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e72249-1c41-4d27-bbb3-5ebc609c717e admin admin] PATH_INFO: `/v3/projects/25c983389ecf447c9ca4bd6ca01b5a32/users/1b8c41cfa6854a528f43e82e0fb26220/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:57.428070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e72249-1c41-4d27-bbb3-5ebc609c717e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c983389ecf447c9ca4bd6ca01b5a32, user_id=1b8c41cfa6854a528f43e82e0fb26220, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:57.431714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e72249-1c41-4d27-bbb3-5ebc609c717e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:57.452439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1006/2024] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:22:57 2022] PUT /identity/v3/projects/25c983389ecf447c9ca4bd6ca01b5a32/users/1b8c41cfa6854a528f43e82e0fb26220/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:57.454280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c19f1-cdae-4fbf-b9ce-f621d8bace87 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:57.454593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c19f1-cdae-4fbf-b9ce-f621d8bace87 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:57.454861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c19f1-cdae-4fbf-b9ce-f621d8bace87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:57.486998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-9b2c19f1-cdae-4fbf-b9ce-f621d8bace87 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:57.488568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9b2c19f1-cdae-4fbf-b9ce-f621d8bace87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:57.526661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e72249-1c41-4d27-bbb3-5ebc609c717e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:57.529356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-59e72249-1c41-4d27-bbb3-5ebc609c717e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:57.595107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:57.609973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1019/2025] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:57.623101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c19f1-cdae-4fbf-b9ce-f621d8bace87 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:57.625101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9b2c19f1-cdae-4fbf-b9ce-f621d8bace87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:57.676206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e72249-1c41-4d27-bbb3-5ebc609c717e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:57.683229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:57.683883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:57.684399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:57.684623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] PATH_INFO: `/v3/users/fe889b1bf81c48239653c9196abdf54e` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:57.686141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe889b1bf81c48239653c9196abdf54e)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:57.688885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:57.755882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe889b1bf81c48239653c9196abdf54e'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:57.756946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe889b1bf81c48239653c9196abdf54e'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:57.766422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe889b1bf81c48239653c9196abdf54e'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:57.767238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c19f1-cdae-4fbf-b9ce-f621d8bace87 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:57.774334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336595ce-819a-4b8d-97dd-ecd673c95f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:57.774966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336595ce-819a-4b8d-97dd-ecd673c95f91 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:57.775243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336595ce-819a-4b8d-97dd-ecd673c95f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:57.779304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336595ce-819a-4b8d-97dd-ecd673c95f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:57.781497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe889b1bf81c48239653c9196abdf54e'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:57.785804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ba109818-753b-4200-a8c5-0293d05c26c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe889b1bf81c48239653c9196abdf54e'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:57.816048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1007/2026] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:57 2022] DELETE /identity/v3/users/fe889b1bf81c48239653c9196abdf54e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:57.818206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336595ce-819a-4b8d-97dd-ecd673c95f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:57.819736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336595ce-819a-4b8d-97dd-ecd673c95f91 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:57.823193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748982e-970a-450f-92cb-1b6ecbf2bc34 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:57.828678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748982e-970a-450f-92cb-1b6ecbf2bc34 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:57.830380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748982e-970a-450f-92cb-1b6ecbf2bc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:57.832789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1020/2027] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:57.878204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:57.913063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-b748982e-970a-450f-92cb-1b6ecbf2bc34 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:57.914964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b748982e-970a-450f-92cb-1b6ecbf2bc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:58.010044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1008/2028] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:58.031221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336595ce-819a-4b8d-97dd-ecd673c95f91 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:58.043068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336595ce-819a-4b8d-97dd-ecd673c95f91 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:58.166327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645e79-6789-43c2-88f5-9acd4fc5331c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:58.167662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645e79-6789-43c2-88f5-9acd4fc5331c service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:58.171762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645e79-6789-43c2-88f5-9acd4fc5331c service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:58.175270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645e79-6789-43c2-88f5-9acd4fc5331c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:58.196034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f645e79-6789-43c2-88f5-9acd4fc5331c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:58.197495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f645e79-6789-43c2-88f5-9acd4fc5331c service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:58.227724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1021/2029] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:59.071383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748982e-970a-450f-92cb-1b6ecbf2bc34 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:59.073579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b748982e-970a-450f-92cb-1b6ecbf2bc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:59.091179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748982e-970a-450f-92cb-1b6ecbf2bc34 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:59.098738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:59.100047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:59.100552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:59.102521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] PATH_INFO: `/v3/users/35153b38b81945fd900769a534210fff` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:59.105044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=35153b38b81945fd900769a534210fff)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:59.108057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:59.187186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35153b38b81945fd900769a534210fff'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.188390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35153b38b81945fd900769a534210fff'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.216526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35153b38b81945fd900769a534210fff'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.225597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35153b38b81945fd900769a534210fff'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.226258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-1e57a19c-8d3c-4113-8706-276a7ca203cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35153b38b81945fd900769a534210fff'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.248023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645e79-6789-43c2-88f5-9acd4fc5331c service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:59.249667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1f645e79-6789-43c2-88f5-9acd4fc5331c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:59.259261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1009/2030] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:59 2022] DELETE /identity/v3/users/35153b38b81945fd900769a534210fff => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:22:59.261164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc63bb4-bfbd-48df-a6b4-5d19cae3b014 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:59.261863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc63bb4-bfbd-48df-a6b4-5d19cae3b014 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:59.263642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc63bb4-bfbd-48df-a6b4-5d19cae3b014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:59.269545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645e79-6789-43c2-88f5-9acd4fc5331c service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:59.314303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ddc63bb4-bfbd-48df-a6b4-5d19cae3b014 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:59.316106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ddc63bb4-bfbd-48df-a6b4-5d19cae3b014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:59.416593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1010/2031] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:59.418244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b461c-5976-4369-bba2-cc34cf0f3c6c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:59.418735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b461c-5976-4369-bba2-cc34cf0f3c6c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:59.419151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b461c-5976-4369-bba2-cc34cf0f3c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:59.445694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:59.446436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:59.446720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:59.447152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] PATH_INFO: `/v3/projects/5a666c9d4266471088ffc6a45d2066fb` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:59.448359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a666c9d4266471088ffc6a45d2066fb)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:59.451132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:59.493056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-182b461c-5976-4369-bba2-cc34cf0f3c6c None None] MFA Rules not processed for user `1b8c41cfa6854a528f43e82e0fb26220`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:22:59.494584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-182b461c-5976-4369-bba2-cc34cf0f3c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:59.500908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:59.529717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:22:59.531414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a666c9d4266471088ffc6a45d2066fb', 'request_id': 'req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.551917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a666c9d4266471088ffc6a45d2066fb', 'request_id': 'req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.574130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a666c9d4266471088ffc6a45d2066fb', 'request_id': 'req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.591805 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1011/2032] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:22:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:22:59.592713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b461c-5976-4369-bba2-cc34cf0f3c6c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:59.594164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-182b461c-5976-4369-bba2-cc34cf0f3c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:59.615079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] The token cache is being invalidate because project 5a666c9d4266471088ffc6a45d2066fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:22:59.615616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.616458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:22:59.618170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1022/2033] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:22:59 2022] DELETE /identity/v3/projects/5a666c9d4266471088ffc6a45d2066fb => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:22:59.619339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:59.620926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:59.700894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b461c-5976-4369-bba2-cc34cf0f3c6c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:59.763171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b88e7-0ee8-4685-b8b5-d2f2f2fb787e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:59.783731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:59.784366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:59.784644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:59.784916 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:59.785865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:22:59.788008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:59.793014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:22:59.796390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:22:59.797044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:22:59.801876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:59.910478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:59.912851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:59.927836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1023/2034] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:59.928866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:22:59.942376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:22:59.942934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:22:59.944552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:22:59.958945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1012/2035] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:22:59.973275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd79258-55f2-4772-90a9-aa3624c4604f service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:22:59.999293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:00.000071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:00.001205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:00.001575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] PATH_INFO: `/v3/users/eabf7aa514ad462e8b155bf19b5cf6d7` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:00.003251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eabf7aa514ad462e8b155bf19b5cf6d7)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:00.005541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:00.099411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eabf7aa514ad462e8b155bf19b5cf6d7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.100188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eabf7aa514ad462e8b155bf19b5cf6d7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.115586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eabf7aa514ad462e8b155bf19b5cf6d7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.116185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eabf7aa514ad462e8b155bf19b5cf6d7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.136417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-56ee4687-a4fc-46b7-8518-1bf48dd162cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eabf7aa514ad462e8b155bf19b5cf6d7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.166045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1024/2036] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:22:59 2022] DELETE /identity/v3/users/eabf7aa514ad462e8b155bf19b5cf6d7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:00.443917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:00.445415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:00.471367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f0254-51fe-421d-9d95-0a8b5d661698 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:00.502659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:00.630359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:00.630998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:00.631474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:00.631550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] PATH_INFO: `/v3/projects/3cfcac2b380940e99b568076438839b4` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:00.634690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cfcac2b380940e99b568076438839b4)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:00.636773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:00.735420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cfcac2b380940e99b568076438839b4', 'request_id': 'req-65c8c036-ed23-4373-a54b-2ba5941a00dc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.743781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cfcac2b380940e99b568076438839b4', 'request_id': 'req-65c8c036-ed23-4373-a54b-2ba5941a00dc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.752256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cfcac2b380940e99b568076438839b4', 'request_id': 'req-65c8c036-ed23-4373-a54b-2ba5941a00dc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.765754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] The token cache is being invalidate because project 3cfcac2b380940e99b568076438839b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:00.766048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.766704 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:00.768317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1013/2037] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:00 2022] DELETE /identity/v3/projects/3cfcac2b380940e99b568076438839b4 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:00.773195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a7446-62ac-4a51-be08-416b4ae76242 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:00.774686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a7446-62ac-4a51-be08-416b4ae76242 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:00.775256 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a7446-62ac-4a51-be08-416b4ae76242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:00.784264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:00.799623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ab8a7446-62ac-4a51-be08-416b4ae76242 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:00.801214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ab8a7446-62ac-4a51-be08-416b4ae76242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:00.855676 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:00.857501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:00.916371 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1025/2038] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:00.917785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5cb13-5887-469f-9be7-3124a7029ab4 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:00.918068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5cb13-5887-469f-9be7-3124a7029ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:00.918336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5cb13-5887-469f-9be7-3124a7029ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:00.939985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-57d5cb13-5887-469f-9be7-3124a7029ab4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:00.941189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-57d5cb13-5887-469f-9be7-3124a7029ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:00.946051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8c036-ed23-4373-a54b-2ba5941a00dc admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:00.950379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1026/2039] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:00.952189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:00.952934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:00.953449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:00.953580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] PATH_INFO: `/v3/projects/2cb4082572474d1c9635458745d57b01` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:00.954508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5cb13-5887-469f-9be7-3124a7029ab4 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:00.954863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb4082572474d1c9635458745d57b01)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:00.956849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:00.964120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5cb13-5887-469f-9be7-3124a7029ab4 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:00.969267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d23e0c-15f3-4a0c-b845-8c0c2382f442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:00.970077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d23e0c-15f3-4a0c-b845-8c0c2382f442 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:00.970425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d23e0c-15f3-4a0c-b845-8c0c2382f442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:00.970770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d23e0c-15f3-4a0c-b845-8c0c2382f442 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:00.971491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d23e0c-15f3-4a0c-b845-8c0c2382f442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:23:00.971861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d23e0c-15f3-4a0c-b845-8c0c2382f442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:00.973497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d23e0c-15f3-4a0c-b845-8c0c2382f442 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:00.986235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1027/2040] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:23:00.990951 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d23e0c-15f3-4a0c-b845-8c0c2382f442 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.002359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d23e0c-15f3-4a0c-b845-8c0c2382f442 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.009852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:01.018757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb4082572474d1c9635458745d57b01', 'request_id': 'req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.030675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb4082572474d1c9635458745d57b01', 'request_id': 'req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.040227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb4082572474d1c9635458745d57b01', 'request_id': 'req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.053081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] The token cache is being invalidate because project 2cb4082572474d1c9635458745d57b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:01.053579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.054333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.056496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1014/2041] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:00 2022] DELETE /identity/v3/projects/2cb4082572474d1c9635458745d57b01 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:01.061470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.064030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:01.080984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17994451-706d-43d8-b5af-e05161b7aeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.081954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17994451-706d-43d8-b5af-e05161b7aeb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.082237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17994451-706d-43d8-b5af-e05161b7aeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.082512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17994451-706d-43d8-b5af-e05161b7aeb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.083150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17994451-706d-43d8-b5af-e05161b7aeb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.087347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17994451-706d-43d8-b5af-e05161b7aeb1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.123981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1028/2042] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:00 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:01.126297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17994451-706d-43d8-b5af-e05161b7aeb1 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.128063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-17994451-706d-43d8-b5af-e05161b7aeb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:01.156144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16daa-4a86-4628-bb02-9c4ee2e28b9e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.224451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17994451-706d-43d8-b5af-e05161b7aeb1 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.229267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b3e7a-62eb-457c-b9a6-2f4e4604b2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.229939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b3e7a-62eb-457c-b9a6-2f4e4604b2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.230215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b3e7a-62eb-457c-b9a6-2f4e4604b2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.230484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b3e7a-62eb-457c-b9a6-2f4e4604b2c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.231162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b3e7a-62eb-457c-b9a6-2f4e4604b2c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.233110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b3e7a-62eb-457c-b9a6-2f4e4604b2c5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.242168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.246216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.246534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.246534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] PATH_INFO: `/v3/users/4611e05ead4b407c8b2367d2ed76d69d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.246534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4611e05ead4b407c8b2367d2ed76d69d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.250655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.251726 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:01.266717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1029/2043] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:01 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:01.267466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b3e7a-62eb-457c-b9a6-2f4e4604b2c5 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.281662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b3e7a-62eb-457c-b9a6-2f4e4604b2c5 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.288158 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01062086-a73b-464c-9785-515f42c272a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.288992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01062086-a73b-464c-9785-515f42c272a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.289276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01062086-a73b-464c-9785-515f42c272a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.289590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01062086-a73b-464c-9785-515f42c272a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.290301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01062086-a73b-464c-9785-515f42c272a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.292012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01062086-a73b-464c-9785-515f42c272a5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.304820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1030/2044] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:01.310048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01062086-a73b-464c-9785-515f42c272a5 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.317545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4611e05ead4b407c8b2367d2ed76d69d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.318402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4611e05ead4b407c8b2367d2ed76d69d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.320342 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01062086-a73b-464c-9785-515f42c272a5 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.325145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bd68e-06c3-4303-8ce1-2dde2ee569aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.325790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bd68e-06c3-4303-8ce1-2dde2ee569aa admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.326073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bd68e-06c3-4303-8ce1-2dde2ee569aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.326345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bd68e-06c3-4303-8ce1-2dde2ee569aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.327292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bd68e-06c3-4303-8ce1-2dde2ee569aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.328585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bd68e-06c3-4303-8ce1-2dde2ee569aa admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.329587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4611e05ead4b407c8b2367d2ed76d69d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.339338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1031/2045] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:23:01.340399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4611e05ead4b407c8b2367d2ed76d69d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.341190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6d44c598-7f29-42e8-880c-e427af28f032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4611e05ead4b407c8b2367d2ed76d69d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.344374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bd68e-06c3-4303-8ce1-2dde2ee569aa admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.346930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-220bd68e-06c3-4303-8ce1-2dde2ee569aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:01.358082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:01.362885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bd68e-06c3-4303-8ce1-2dde2ee569aa admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.365157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1015/2046] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:01 2022] DELETE /identity/v3/users/4611e05ead4b407c8b2367d2ed76d69d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:01.428181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766be246-aaad-4739-b5e5-e54a3b722e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.428858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766be246-aaad-4739-b5e5-e54a3b722e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.429209 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766be246-aaad-4739-b5e5-e54a3b722e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.429425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766be246-aaad-4739-b5e5-e54a3b722e66 admin admin] PATH_INFO: `/v3/projects/973809a885644f4199c8bc707b52a829/users/61e0b0ca8f46467d9408388d273341b9/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.448134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766be246-aaad-4739-b5e5-e54a3b722e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=973809a885644f4199c8bc707b52a829, user_id=61e0b0ca8f46467d9408388d273341b9, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.451176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766be246-aaad-4739-b5e5-e54a3b722e66 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.466996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1032/2047] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:01 2022] PUT /identity/v3/projects/973809a885644f4199c8bc707b52a829/users/61e0b0ca8f46467d9408388d273341b9/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:01.478782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7e203-8d9a-4c5d-bfc5-10a652d94a42 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.479256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7e203-8d9a-4c5d-bfc5-10a652d94a42 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.479659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7e203-8d9a-4c5d-bfc5-10a652d94a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.509998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-69b7e203-8d9a-4c5d-bfc5-10a652d94a42 None None] MFA Rules not processed for user `61e0b0ca8f46467d9408388d273341b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:01.511483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-69b7e203-8d9a-4c5d-bfc5-10a652d94a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:01.627366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1016/2048] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:23:01.632225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766be246-aaad-4739-b5e5-e54a3b722e66 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.723884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7e203-8d9a-4c5d-bfc5-10a652d94a42 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.725380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-69b7e203-8d9a-4c5d-bfc5-10a652d94a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:01.735890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766be246-aaad-4739-b5e5-e54a3b722e66 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.741076 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cd65c-a4eb-41fe-bb9c-c34597a8ea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.741726 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cd65c-a4eb-41fe-bb9c-c34597a8ea81 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.742008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cd65c-a4eb-41fe-bb9c-c34597a8ea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.742280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412cd65c-a4eb-41fe-bb9c-c34597a8ea81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.742963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cd65c-a4eb-41fe-bb9c-c34597a8ea81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.744530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7e203-8d9a-4c5d-bfc5-10a652d94a42 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.745242 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412cd65c-a4eb-41fe-bb9c-c34597a8ea81 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.749884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.750824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.750936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.751128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] PATH_INFO: `/v3/projects/a9cc5feeb3cb4edcbbff127a5b6a81af` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.752244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9cc5feeb3cb4edcbbff127a5b6a81af)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.755213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.779152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1033/2049] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:01 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:01.783117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cd65c-a4eb-41fe-bb9c-c34597a8ea81 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.813419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9cc5feeb3cb4edcbbff127a5b6a81af', 'request_id': 'req-f2904498-b19c-4a09-b0fe-defd887839f0'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.822581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9cc5feeb3cb4edcbbff127a5b6a81af', 'request_id': 'req-f2904498-b19c-4a09-b0fe-defd887839f0'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.830722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9cc5feeb3cb4edcbbff127a5b6a81af', 'request_id': 'req-f2904498-b19c-4a09-b0fe-defd887839f0'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.835726 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:01.845814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] The token cache is being invalidate because project a9cc5feeb3cb4edcbbff127a5b6a81af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:01.846108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.846787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f2904498-b19c-4a09-b0fe-defd887839f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:01.848413 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1017/2050] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:01 2022] DELETE /identity/v3/projects/a9cc5feeb3cb4edcbbff127a5b6a81af => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:01.853950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bf0fd-1d8a-4966-928e-3da1f1aac486 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.854232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bf0fd-1d8a-4966-928e-3da1f1aac486 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.854669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bf0fd-1d8a-4966-928e-3da1f1aac486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.857552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412cd65c-a4eb-41fe-bb9c-c34597a8ea81 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.881659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ae8bf0fd-1d8a-4966-928e-3da1f1aac486 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:01.882888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ae8bf0fd-1d8a-4966-928e-3da1f1aac486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:01.923641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86384d98-4535-4ce9-bbe0-4d351e2d90b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.924274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86384d98-4535-4ce9-bbe0-4d351e2d90b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.924554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86384d98-4535-4ce9-bbe0-4d351e2d90b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.924822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86384d98-4535-4ce9-bbe0-4d351e2d90b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.925443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86384d98-4535-4ce9-bbe0-4d351e2d90b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.927503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86384d98-4535-4ce9-bbe0-4d351e2d90b2 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.946166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1018/2051] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:01.949573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1034/2052] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:01 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:01.953396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86384d98-4535-4ce9-bbe0-4d351e2d90b2 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.954122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3854f-48b5-491c-912a-87a61f999a57 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.954440 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3854f-48b5-491c-912a-87a61f999a57 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.954736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3854f-48b5-491c-912a-87a61f999a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.955085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-86384d98-4535-4ce9-bbe0-4d351e2d90b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:01.967187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86384d98-4535-4ce9-bbe0-4d351e2d90b2 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:01.971852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4f117-5607-487f-a5b4-34332347c844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:01.972437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4f117-5607-487f-a5b4-34332347c844 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:01.972715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4f117-5607-487f-a5b4-34332347c844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:01.972985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4f117-5607-487f-a5b4-34332347c844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:01.973649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4f117-5607-487f-a5b4-34332347c844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:01.975229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc4f117-5607-487f-a5b4-34332347c844 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:01.975767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-daa3854f-48b5-491c-912a-87a61f999a57 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:01.977103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-daa3854f-48b5-491c-912a-87a61f999a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:01.985493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1019/2053] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:23:01.986070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1035/2054] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:01.989716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3854f-48b5-491c-912a-87a61f999a57 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.989839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4f117-5607-487f-a5b4-34332347c844 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:01.995612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:01.999209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3854f-48b5-491c-912a-87a61f999a57 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.000224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4f117-5607-487f-a5b4-34332347c844 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.004934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0460d-4a88-4bd4-b864-ddecfc02a593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.005564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0460d-4a88-4bd4-b864-ddecfc02a593 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.006079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060982ef-b7ef-45fc-8d14-7bd27a354fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.006203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0460d-4a88-4bd4-b864-ddecfc02a593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.006639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0460d-4a88-4bd4-b864-ddecfc02a593 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.006742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060982ef-b7ef-45fc-8d14-7bd27a354fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.006920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060982ef-b7ef-45fc-8d14-7bd27a354fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.007187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060982ef-b7ef-45fc-8d14-7bd27a354fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.007335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0460d-4a88-4bd4-b864-ddecfc02a593 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:23:02.007815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060982ef-b7ef-45fc-8d14-7bd27a354fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.007923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0460d-4a88-4bd4-b864-ddecfc02a593 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.009387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060982ef-b7ef-45fc-8d14-7bd27a354fd1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.009502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0460d-4a88-4bd4-b864-ddecfc02a593 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.019868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1036/2055] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:23:02.022475 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1020/2056] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:23:02.023963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060982ef-b7ef-45fc-8d14-7bd27a354fd1 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.026350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0460d-4a88-4bd4-b864-ddecfc02a593 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.034261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060982ef-b7ef-45fc-8d14-7bd27a354fd1 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.037845 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0460d-4a88-4bd4-b864-ddecfc02a593 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.038794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce2dd2-73d7-4729-ba0a-bdc3a43a3708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.039825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce2dd2-73d7-4729-ba0a-bdc3a43a3708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.040099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce2dd2-73d7-4729-ba0a-bdc3a43a3708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.040365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce2dd2-73d7-4729-ba0a-bdc3a43a3708 admin admin] PATH_INFO: `/v3/projects/4d43e7b9c9f2433bae823d9c6d13e012/users/b32844f8eef84160b6eccfa36b2f45ae/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.042602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082445a4-28c5-439e-a3eb-92b7c89eac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.043093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082445a4-28c5-439e-a3eb-92b7c89eac39 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.043374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082445a4-28c5-439e-a3eb-92b7c89eac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.044205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082445a4-28c5-439e-a3eb-92b7c89eac39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.044833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082445a4-28c5-439e-a3eb-92b7c89eac39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.046763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082445a4-28c5-439e-a3eb-92b7c89eac39 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.055452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce2dd2-73d7-4729-ba0a-bdc3a43a3708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d43e7b9c9f2433bae823d9c6d13e012, user_id=b32844f8eef84160b6eccfa36b2f45ae, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.058107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce2dd2-73d7-4729-ba0a-bdc3a43a3708 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.072150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1037/2057] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:02 2022] PUT /identity/v3/projects/4d43e7b9c9f2433bae823d9c6d13e012/users/b32844f8eef84160b6eccfa36b2f45ae/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:02.073905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1021/2058] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:02 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:23:02.078389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce2dd2-73d7-4729-ba0a-bdc3a43a3708 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.082623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad1ff6-8165-4215-bdbc-3aff3abcd975 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.083077 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad1ff6-8165-4215-bdbc-3aff3abcd975 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.083352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad1ff6-8165-4215-bdbc-3aff3abcd975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.110939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-3aad1ff6-8165-4215-bdbc-3aff3abcd975 None None] MFA Rules not processed for user `b32844f8eef84160b6eccfa36b2f45ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:02.112149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3aad1ff6-8165-4215-bdbc-3aff3abcd975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:02.140294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce2dd2-73d7-4729-ba0a-bdc3a43a3708 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.178901 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:02.180144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1022/2059] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:23:02.184940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad1ff6-8165-4215-bdbc-3aff3abcd975 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.196030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e3323-9e7f-4983-85cf-42b61952370b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.196634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e3323-9e7f-4983-85cf-42b61952370b admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.196919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e3323-9e7f-4983-85cf-42b61952370b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.197217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e3323-9e7f-4983-85cf-42b61952370b admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.197886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e3323-9e7f-4983-85cf-42b61952370b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.200358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e3323-9e7f-4983-85cf-42b61952370b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.225420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1038/2060] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:02 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:02.229028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e3323-9e7f-4983-85cf-42b61952370b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.240016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e3323-9e7f-4983-85cf-42b61952370b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.244729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbde2c-7436-4b6d-b420-cd859868a5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.245323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbde2c-7436-4b6d-b420-cd859868a5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.245632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbde2c-7436-4b6d-b420-cd859868a5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.245908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbde2c-7436-4b6d-b420-cd859868a5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.246571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbde2c-7436-4b6d-b420-cd859868a5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.248281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbde2c-7436-4b6d-b420-cd859868a5e8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.256326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad1ff6-8165-4215-bdbc-3aff3abcd975 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.259762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1039/2061] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:02.263321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d3b1e-c6aa-4a88-8c5c-24087fd8cc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.263321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d3b1e-c6aa-4a88-8c5c-24087fd8cc82 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.263535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d3b1e-c6aa-4a88-8c5c-24087fd8cc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.263897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d3b1e-c6aa-4a88-8c5c-24087fd8cc82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.264193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbde2c-7436-4b6d-b420-cd859868a5e8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.264588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d3b1e-c6aa-4a88-8c5c-24087fd8cc82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.266538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d3b1e-c6aa-4a88-8c5c-24087fd8cc82 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.274992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbde2c-7436-4b6d-b420-cd859868a5e8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.280270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0079-8f28-4ec7-ab2e-15552f5c5656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.280866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0079-8f28-4ec7-ab2e-15552f5c5656 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.281139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0079-8f28-4ec7-ab2e-15552f5c5656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.281574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0079-8f28-4ec7-ab2e-15552f5c5656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.282196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a0079-8f28-4ec7-ab2e-15552f5c5656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.283776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a0079-8f28-4ec7-ab2e-15552f5c5656 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.296724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1023/2062] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:02 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:23:02.300539 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d3b1e-c6aa-4a88-8c5c-24087fd8cc82 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.307179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1040/2063] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:02.311473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0079-8f28-4ec7-ab2e-15552f5c5656 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.322625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0079-8f28-4ec7-ab2e-15552f5c5656 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.358907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:02.378576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d3b1e-c6aa-4a88-8c5c-24087fd8cc82 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.386417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273ab7-28ac-4b63-ae00-03515ba2cf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.387042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273ab7-28ac-4b63-ae00-03515ba2cf1d admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.387852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273ab7-28ac-4b63-ae00-03515ba2cf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.388191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11273ab7-28ac-4b63-ae00-03515ba2cf1d admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.388758 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11273ab7-28ac-4b63-ae00-03515ba2cf1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.390724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11273ab7-28ac-4b63-ae00-03515ba2cf1d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.396489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe65d26-3485-47a6-bf98-3c8e07f82ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.397357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe65d26-3485-47a6-bf98-3c8e07f82ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.397714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe65d26-3485-47a6-bf98-3c8e07f82ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.398076 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe65d26-3485-47a6-bf98-3c8e07f82ff7 admin admin] PATH_INFO: `/v3/projects/1c2141e33c264250aa7642f46a9b4048/users/776c647ef6d34d989ac12d4e91cc62ab/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.415627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1024/2064] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:02 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:23:02.419412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273ab7-28ac-4b63-ae00-03515ba2cf1d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.422183 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe65d26-3485-47a6-bf98-3c8e07f82ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2141e33c264250aa7642f46a9b4048, user_id=776c647ef6d34d989ac12d4e91cc62ab, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.425161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe65d26-3485-47a6-bf98-3c8e07f82ff7 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.429710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11273ab7-28ac-4b63-ae00-03515ba2cf1d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.435756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49905d-3928-42db-8b0c-41896f7e3efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.436406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49905d-3928-42db-8b0c-41896f7e3efb admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.436669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49905d-3928-42db-8b0c-41896f7e3efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.436939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49905d-3928-42db-8b0c-41896f7e3efb admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.437815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49905d-3928-42db-8b0c-41896f7e3efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.439402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49905d-3928-42db-8b0c-41896f7e3efb admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.442823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1041/2065] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:02 2022] PUT /identity/v3/projects/1c2141e33c264250aa7642f46a9b4048/users/776c647ef6d34d989ac12d4e91cc62ab/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:02.450831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c33b42-886e-4b4c-9aa5-cea9416d1815 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.451197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c33b42-886e-4b4c-9aa5-cea9416d1815 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.451542 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c33b42-886e-4b4c-9aa5-cea9416d1815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.453249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1025/2066] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:02.457300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49905d-3928-42db-8b0c-41896f7e3efb admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.466927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49905d-3928-42db-8b0c-41896f7e3efb admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.471199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c6c33b42-886e-4b4c-9aa5-cea9416d1815 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:02.472388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c6c33b42-886e-4b4c-9aa5-cea9416d1815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:02.525971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b897b0-5aa1-4acb-b4af-0e6168cf4af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.526635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b897b0-5aa1-4acb-b4af-0e6168cf4af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.526823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b897b0-5aa1-4acb-b4af-0e6168cf4af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.527094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b897b0-5aa1-4acb-b4af-0e6168cf4af2 admin admin] PATH_INFO: `/v3/projects/20bd2b69c9414e678ddf5518fd175f80/users/570d0423d43740a096e7276d744430dc/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.532688 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1042/2067] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:23:02.542516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c33b42-886e-4b4c-9aa5-cea9416d1815 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.543935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b897b0-5aa1-4acb-b4af-0e6168cf4af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bd2b69c9414e678ddf5518fd175f80, user_id=570d0423d43740a096e7276d744430dc, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.544052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c6c33b42-886e-4b4c-9aa5-cea9416d1815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:02.546650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b897b0-5aa1-4acb-b4af-0e6168cf4af2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.562011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1026/2068] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:02 2022] PUT /identity/v3/projects/20bd2b69c9414e678ddf5518fd175f80/users/570d0423d43740a096e7276d744430dc/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:02.565777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b897b0-5aa1-4acb-b4af-0e6168cf4af2 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.580569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:02.608558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c33b42-886e-4b4c-9aa5-cea9416d1815 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.635198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b897b0-5aa1-4acb-b4af-0e6168cf4af2 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.641111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087f24d-3ca9-4031-92cf-b0312e408025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.641754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087f24d-3ca9-4031-92cf-b0312e408025 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.642030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087f24d-3ca9-4031-92cf-b0312e408025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.642299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087f24d-3ca9-4031-92cf-b0312e408025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.643008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087f24d-3ca9-4031-92cf-b0312e408025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.644502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087f24d-3ca9-4031-92cf-b0312e408025 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.657541 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1027/2069] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:23:02.658743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087f24d-3ca9-4031-92cf-b0312e408025 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.668547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087f24d-3ca9-4031-92cf-b0312e408025 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.673399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9e28a-8e99-4050-a3c6-a4bd65289ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.674070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9e28a-8e99-4050-a3c6-a4bd65289ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.674243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021c943-dede-4125-9073-6d68149ac9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.674374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9e28a-8e99-4050-a3c6-a4bd65289ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.674604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9e28a-8e99-4050-a3c6-a4bd65289ad2 admin admin] PATH_INFO: `/v3/projects/20bd2b69c9414e678ddf5518fd175f80/users/570d0423d43740a096e7276d744430dc/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.674973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021c943-dede-4125-9073-6d68149ac9df service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.675253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021c943-dede-4125-9073-6d68149ac9df service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.675522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021c943-dede-4125-9073-6d68149ac9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.676507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9e28a-8e99-4050-a3c6-a4bd65289ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20bd2b69c9414e678ddf5518fd175f80, user_id=570d0423d43740a096e7276d744430dc, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.679190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9e28a-8e99-4050-a3c6-a4bd65289ad2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.685231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021c943-dede-4125-9073-6d68149ac9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:02.686673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021c943-dede-4125-9073-6d68149ac9df service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:02.690660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1028/2070] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:02 2022] PUT /identity/v3/projects/20bd2b69c9414e678ddf5518fd175f80/users/570d0423d43740a096e7276d744430dc/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:02.698384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1043/2071] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:23:02.701754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517df52-9275-42d1-a127-7f45496c65dc None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.702047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517df52-9275-42d1-a127-7f45496c65dc None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.702325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517df52-9275-42d1-a127-7f45496c65dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.735500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e517df52-9275-42d1-a127-7f45496c65dc None None] MFA Rules not processed for user `570d0423d43740a096e7276d744430dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:02.736714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e517df52-9275-42d1-a127-7f45496c65dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:02.756073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:02.829789 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1029/2072] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:02.841291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021c943-dede-4125-9073-6d68149ac9df service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:02.842917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3021c943-dede-4125-9073-6d68149ac9df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:02.909619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021c943-dede-4125-9073-6d68149ac9df service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:02.960825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e8dbc-b056-4471-9028-51127ef3afc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:02.961436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e8dbc-b056-4471-9028-51127ef3afc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:02.961743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e8dbc-b056-4471-9028-51127ef3afc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:02.962015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e8dbc-b056-4471-9028-51127ef3afc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:02.963751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7f0e8dbc-b056-4471-9028-51127ef3afc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:03.030915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e8dbc-b056-4471-9028-51127ef3afc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:03.032386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e8dbc-b056-4471-9028-51127ef3afc1 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:03.042514 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1044/2073] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:03.449412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf2a4f-12f9-46f3-83fa-e1adf413ce1c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:03.449811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf2a4f-12f9-46f3-83fa-e1adf413ce1c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:03.450152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf2a4f-12f9-46f3-83fa-e1adf413ce1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:03.480767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7ebf2a4f-12f9-46f3-83fa-e1adf413ce1c None None] MFA Rules not processed for user `776c647ef6d34d989ac12d4e91cc62ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:03.482610 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7ebf2a4f-12f9-46f3-83fa-e1adf413ce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:03.556149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1030/2074] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:03.568189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e8dbc-b056-4471-9028-51127ef3afc1 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:03.572355 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:03.579630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e8dbc-b056-4471-9028-51127ef3afc1 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:03.584662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c570d2c-ef1c-437e-bf65-32d428fc70ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:03.584836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c570d2c-ef1c-437e-bf65-32d428fc70ba service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:03.585060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c570d2c-ef1c-437e-bf65-32d428fc70ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:03.585327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c570d2c-ef1c-437e-bf65-32d428fc70ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:03.594943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c570d2c-ef1c-437e-bf65-32d428fc70ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:03.596506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c570d2c-ef1c-437e-bf65-32d428fc70ba service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:03.607058 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1045/2075] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:04.586666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf2a4f-12f9-46f3-83fa-e1adf413ce1c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:04.597288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf2a4f-12f9-46f3-83fa-e1adf413ce1c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:04.600767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef547a-75e2-4531-9e1e-a3af0e6e710e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:04.601444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef547a-75e2-4531-9e1e-a3af0e6e710e service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:04.601742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef547a-75e2-4531-9e1e-a3af0e6e710e service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:04.602063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef547a-75e2-4531-9e1e-a3af0e6e710e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:04.666722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef547a-75e2-4531-9e1e-a3af0e6e710e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:04.668439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef547a-75e2-4531-9e1e-a3af0e6e710e service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:04.678950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1031/2076] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:04.918141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa018703-ca11-4d87-92c2-3f1bed757f99 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:04.918645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa018703-ca11-4d87-92c2-3f1bed757f99 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:04.918732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa018703-ca11-4d87-92c2-3f1bed757f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:04.938819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-fa018703-ca11-4d87-92c2-3f1bed757f99 None None] MFA Rules not processed for user `1b8c41cfa6854a528f43e82e0fb26220`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:04.940085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fa018703-ca11-4d87-92c2-3f1bed757f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:05.000114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f32c9-d5f1-473d-b61c-ddf69670d175 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:05.000433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f32c9-d5f1-473d-b61c-ddf69670d175 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:05.000709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f32c9-d5f1-473d-b61c-ddf69670d175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:05.011495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1046/2077] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:05.024461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-098f32c9-d5f1-473d-b61c-ddf69670d175 None None] MFA Rules not processed for user `776c647ef6d34d989ac12d4e91cc62ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:05.025804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-098f32c9-d5f1-473d-b61c-ddf69670d175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:05.033961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1032/2078] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:05.286051 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa018703-ca11-4d87-92c2-3f1bed757f99 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:05.297130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa018703-ca11-4d87-92c2-3f1bed757f99 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:05.301038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7664651-a2cd-45b9-a9ae-13983ecd34fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:05.301684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7664651-a2cd-45b9-a9ae-13983ecd34fb service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:05.301983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7664651-a2cd-45b9-a9ae-13983ecd34fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:05.302266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7664651-a2cd-45b9-a9ae-13983ecd34fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:05.311958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7664651-a2cd-45b9-a9ae-13983ecd34fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:05.313397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7664651-a2cd-45b9-a9ae-13983ecd34fb service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:05.324107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1047/2079] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:07.601182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f32c9-d5f1-473d-b61c-ddf69670d175 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:07.602708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-098f32c9-d5f1-473d-b61c-ddf69670d175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:07.643666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:07.668623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f32c9-d5f1-473d-b61c-ddf69670d175 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:07.673766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:07.674376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:07.674653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:07.674924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] PATH_INFO: `/v3/users/fa0c6e57932b4f3fa952a69e3cb9203a` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:07.676020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa0c6e57932b4f3fa952a69e3cb9203a)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:07.678056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:07.722586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa0c6e57932b4f3fa952a69e3cb9203a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:07.723229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa0c6e57932b4f3fa952a69e3cb9203a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:07.730395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa0c6e57932b4f3fa952a69e3cb9203a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:07.738005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa0c6e57932b4f3fa952a69e3cb9203a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:07.738648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa0c6e57932b4f3fa952a69e3cb9203a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:07.755305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1033/2080] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:07 2022] DELETE /identity/v3/users/fa0c6e57932b4f3fa952a69e3cb9203a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:07.759738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3168235-bce1-46cf-b8ef-13cdc3e2afb2 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:07.760247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3168235-bce1-46cf-b8ef-13cdc3e2afb2 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:07.760349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3168235-bce1-46cf-b8ef-13cdc3e2afb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:07.778616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b3168235-bce1-46cf-b8ef-13cdc3e2afb2 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:07.780346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b3168235-bce1-46cf-b8ef-13cdc3e2afb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:07.837844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1048/2081] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:07.848111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:07.849531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:07.922601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa61fe-e676-46e6-9f6f-cd21d473c4f4 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:07.928791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbf505-adf9-453e-a496-64e6ab0b81af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:07.929542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbf505-adf9-453e-a496-64e6ab0b81af service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:07.929946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbf505-adf9-453e-a496-64e6ab0b81af service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:07.930318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbf505-adf9-453e-a496-64e6ab0b81af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:07.940390 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbf505-adf9-453e-a496-64e6ab0b81af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:07.942279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbf505-adf9-453e-a496-64e6ab0b81af service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:07.953000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1034/2082] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:08.249100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3168235-bce1-46cf-b8ef-13cdc3e2afb2 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:08.249100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b3168235-bce1-46cf-b8ef-13cdc3e2afb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:08.257917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3168235-bce1-46cf-b8ef-13cdc3e2afb2 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:08.264853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:08.265546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:08.265829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:08.266216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] PATH_INFO: `/v3/projects/032eacedb6de402cb9cca2453f68e9a7` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:08.268274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] RBAC: Authorizing `identity:delete_project(project_id=032eacedb6de402cb9cca2453f68e9a7)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:08.270379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:08.328778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032eacedb6de402cb9cca2453f68e9a7', 'request_id': 'req-283e5783-39ea-4468-985c-9e2af8e8bade'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.336112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032eacedb6de402cb9cca2453f68e9a7', 'request_id': 'req-283e5783-39ea-4468-985c-9e2af8e8bade'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.343490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032eacedb6de402cb9cca2453f68e9a7', 'request_id': 'req-283e5783-39ea-4468-985c-9e2af8e8bade'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.354591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] The token cache is being invalidate because project 032eacedb6de402cb9cca2453f68e9a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:08.354880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.355548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.359989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1049/2083] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:08 2022] DELETE /identity/v3/projects/032eacedb6de402cb9cca2453f68e9a7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:08.360979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbf505-adf9-453e-a496-64e6ab0b81af service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:08.371382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbf505-adf9-453e-a496-64e6ab0b81af service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:08.429087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:08.437170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:08.438011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:08.438738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:08.439373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] PATH_INFO: `/v3/users/cd05c7d44369424080a25977e0a92019` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:08.441122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd05c7d44369424080a25977e0a92019)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:08.444037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:08.492041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd05c7d44369424080a25977e0a92019'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.492616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd05c7d44369424080a25977e0a92019'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.505807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd05c7d44369424080a25977e0a92019'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.514587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd05c7d44369424080a25977e0a92019'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.515236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-def4aa10-97dd-4833-9810-729ae703c20b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd05c7d44369424080a25977e0a92019'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.532995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1035/2084] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:08 2022] DELETE /identity/v3/users/cd05c7d44369424080a25977e0a92019 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:08.805727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:08.807148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:08.833485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:08.838012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e5783-39ea-4468-985c-9e2af8e8bade admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:08.896018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:08.896652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:08.896958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:08.897244 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] PATH_INFO: `/v3/projects/8cf48a5a7e2c4ffd85d97ffc5c581613` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:08.898384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cf48a5a7e2c4ffd85d97ffc5c581613)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:08.900347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:08.945016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cf48a5a7e2c4ffd85d97ffc5c581613', 'request_id': 'req-38e016f9-c581-4e6d-8906-796ca2f0143a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.952606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cf48a5a7e2c4ffd85d97ffc5c581613', 'request_id': 'req-38e016f9-c581-4e6d-8906-796ca2f0143a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.958821 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cf48a5a7e2c4ffd85d97ffc5c581613', 'request_id': 'req-38e016f9-c581-4e6d-8906-796ca2f0143a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.969484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] The token cache is being invalidate because project 8cf48a5a7e2c4ffd85d97ffc5c581613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:08.969795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.970420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38e016f9-c581-4e6d-8906-796ca2f0143a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:08.971919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1050/2085] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:08 2022] DELETE /identity/v3/projects/8cf48a5a7e2c4ffd85d97ffc5c581613 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:08.976855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7c666-64e6-402f-93cb-5eee5fc18e71 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:08.977333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7c666-64e6-402f-93cb-5eee5fc18e71 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:08.977416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7c666-64e6-402f-93cb-5eee5fc18e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:08.997936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a3b7c666-64e6-402f-93cb-5eee5fc18e71 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:08.999121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a3b7c666-64e6-402f-93cb-5eee5fc18e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:09.064306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1036/2086] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:09.067504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87dad3-9f8e-42f7-9f7f-a5ac586dd912 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.067784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87dad3-9f8e-42f7-9f7f-a5ac586dd912 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.068053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87dad3-9f8e-42f7-9f7f-a5ac586dd912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.087716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-0c87dad3-9f8e-42f7-9f7f-a5ac586dd912 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:09.088903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0c87dad3-9f8e-42f7-9f7f-a5ac586dd912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:09.097407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1051/2087] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:09.101502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7c666-64e6-402f-93cb-5eee5fc18e71 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.111135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7c666-64e6-402f-93cb-5eee5fc18e71 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.115638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f5ca4-a7b1-41ea-bf5c-836b0468f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.116233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f5ca4-a7b1-41ea-bf5c-836b0468f0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.116505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f5ca4-a7b1-41ea-bf5c-836b0468f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.116778 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f5ca4-a7b1-41ea-bf5c-836b0468f0a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.117404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f5ca4-a7b1-41ea-bf5c-836b0468f0a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:23:09.117731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f5ca4-a7b1-41ea-bf5c-836b0468f0a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.119305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f5ca4-a7b1-41ea-bf5c-836b0468f0a9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.130822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1037/2088] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:23:09.135182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87dad3-9f8e-42f7-9f7f-a5ac586dd912 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.144913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87dad3-9f8e-42f7-9f7f-a5ac586dd912 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.149917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3a107-1bf4-48fa-b3bd-646e2549c7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.150574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3a107-1bf4-48fa-b3bd-646e2549c7ac admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.150919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3a107-1bf4-48fa-b3bd-646e2549c7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.151257 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3a107-1bf4-48fa-b3bd-646e2549c7ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.151989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3a107-1bf4-48fa-b3bd-646e2549c7ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.154559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3a107-1bf4-48fa-b3bd-646e2549c7ac admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.182689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1052/2089] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:09 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:09.187334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f5ca4-a7b1-41ea-bf5c-836b0468f0a9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.197663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f5ca4-a7b1-41ea-bf5c-836b0468f0a9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.253206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b96a9-fc5f-4cf9-8e0d-dd81ca1cd48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.253766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b96a9-fc5f-4cf9-8e0d-dd81ca1cd48f admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.254042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b96a9-fc5f-4cf9-8e0d-dd81ca1cd48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.254310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b96a9-fc5f-4cf9-8e0d-dd81ca1cd48f admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.254925 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b96a9-fc5f-4cf9-8e0d-dd81ca1cd48f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.256818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b96a9-fc5f-4cf9-8e0d-dd81ca1cd48f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.268166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:09.284551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1038/2090] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:09 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:23:09.288545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3a107-1bf4-48fa-b3bd-646e2549c7ac admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.298798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3a107-1bf4-48fa-b3bd-646e2549c7ac admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.304216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445f434-9731-484d-b6c7-405c539c27ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.304826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445f434-9731-484d-b6c7-405c539c27ae admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.305233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445f434-9731-484d-b6c7-405c539c27ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.305529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445f434-9731-484d-b6c7-405c539c27ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.306179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445f434-9731-484d-b6c7-405c539c27ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.307832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445f434-9731-484d-b6c7-405c539c27ae admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.322287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1053/2091] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:09.323943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b96a9-fc5f-4cf9-8e0d-dd81ca1cd48f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.327309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445f434-9731-484d-b6c7-405c539c27ae admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.329238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d445f434-9731-484d-b6c7-405c539c27ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:09.334834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b96a9-fc5f-4cf9-8e0d-dd81ca1cd48f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.340414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205adc2-d5da-4401-b54d-0fe6a0a1c56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.341023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205adc2-d5da-4401-b54d-0fe6a0a1c56d admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.341333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205adc2-d5da-4401-b54d-0fe6a0a1c56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.341643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205adc2-d5da-4401-b54d-0fe6a0a1c56d admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.342293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205adc2-d5da-4401-b54d-0fe6a0a1c56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.342494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445f434-9731-484d-b6c7-405c539c27ae admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.343914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205adc2-d5da-4401-b54d-0fe6a0a1c56d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.352118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.352635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.352759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.352974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] PATH_INFO: `/v3/users/61e0b0ca8f46467d9408388d273341b9` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.354302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e0b0ca8f46467d9408388d273341b9)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.356240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.357139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1039/2092] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:09.362498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205adc2-d5da-4401-b54d-0fe6a0a1c56d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.373445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205adc2-d5da-4401-b54d-0fe6a0a1c56d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.379362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09944ad4-1c22-48d3-83bd-25b9f736b1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.380096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09944ad4-1c22-48d3-83bd-25b9f736b1ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.380478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09944ad4-1c22-48d3-83bd-25b9f736b1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.380840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09944ad4-1c22-48d3-83bd-25b9f736b1ad admin admin] PATH_INFO: `/v3/projects/afa4428e70ac431882b3012118d5c049/users/53d80bf4a8724d30adfb9f0177b0ed1a/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.400198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09944ad4-1c22-48d3-83bd-25b9f736b1ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa4428e70ac431882b3012118d5c049, user_id=53d80bf4a8724d30adfb9f0177b0ed1a, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.403552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09944ad4-1c22-48d3-83bd-25b9f736b1ad admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.415165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e0b0ca8f46467d9408388d273341b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:09.415869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e0b0ca8f46467d9408388d273341b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:09.418106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:09.420091 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1040/2093] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:09 2022] PUT /identity/v3/projects/afa4428e70ac431882b3012118d5c049/users/53d80bf4a8724d30adfb9f0177b0ed1a/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:09.426072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e0b0ca8f46467d9408388d273341b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:09.427041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e0b0ca8f46467d9408388d273341b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:09.432331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76e0c2-4067-4566-9198-5d89da5eec99 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.433080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76e0c2-4067-4566-9198-5d89da5eec99 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.433801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76e0c2-4067-4566-9198-5d89da5eec99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.436717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b12bb77a-94f7-4bc8-9559-682b3237bf48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e0b0ca8f46467d9408388d273341b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:09.463856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1054/2094] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:09 2022] DELETE /identity/v3/users/61e0b0ca8f46467d9408388d273341b9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:09.468537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb429659-4249-4f4b-9a10-10f15d91664c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.468985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb429659-4249-4f4b-9a10-10f15d91664c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.469436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb429659-4249-4f4b-9a10-10f15d91664c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.470860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9c76e0c2-4067-4566-9198-5d89da5eec99 None None] MFA Rules not processed for user `53d80bf4a8724d30adfb9f0177b0ed1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:09.472241 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9c76e0c2-4067-4566-9198-5d89da5eec99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:09.504962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-bb429659-4249-4f4b-9a10-10f15d91664c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:09.508654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-bb429659-4249-4f4b-9a10-10f15d91664c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:09.586812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1041/2095] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:23:09.589839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76e0c2-4067-4566-9198-5d89da5eec99 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.591238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9c76e0c2-4067-4566-9198-5d89da5eec99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:09.608380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1055/2096] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:09.620681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb429659-4249-4f4b-9a10-10f15d91664c None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.623101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-bb429659-4249-4f4b-9a10-10f15d91664c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:09.718460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76e0c2-4067-4566-9198-5d89da5eec99 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.719444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb429659-4249-4f4b-9a10-10f15d91664c None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.723219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fe957-9ccb-47c8-af7d-b364bdaaa5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.724298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fe957-9ccb-47c8-af7d-b364bdaaa5e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.724298 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fe957-9ccb-47c8-af7d-b364bdaaa5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.724469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db29db-baef-40a9-92ca-4cb32caf2054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.724555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fe957-9ccb-47c8-af7d-b364bdaaa5e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.725029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db29db-baef-40a9-92ca-4cb32caf2054 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.725124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fe957-9ccb-47c8-af7d-b364bdaaa5e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.725314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db29db-baef-40a9-92ca-4cb32caf2054 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.725620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db29db-baef-40a9-92ca-4cb32caf2054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.726990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fe957-9ccb-47c8-af7d-b364bdaaa5e9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.736967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db29db-baef-40a9-92ca-4cb32caf2054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.738475 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db29db-baef-40a9-92ca-4cb32caf2054 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.749294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1056/2097] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:09.770140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1042/2098] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:09 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:09.778520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db29db-baef-40a9-92ca-4cb32caf2054 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.790715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db29db-baef-40a9-92ca-4cb32caf2054 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.851737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e64898-c6b2-459b-9f37-d2edaa12e965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.852384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e64898-c6b2-459b-9f37-d2edaa12e965 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.852713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e64898-c6b2-459b-9f37-d2edaa12e965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.853068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e64898-c6b2-459b-9f37-d2edaa12e965 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.853753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e64898-c6b2-459b-9f37-d2edaa12e965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.855654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e64898-c6b2-459b-9f37-d2edaa12e965 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.879965 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:09.886601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1057/2099] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:09 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:23:09.891275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fe957-9ccb-47c8-af7d-b364bdaaa5e9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.912107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fe957-9ccb-47c8-af7d-b364bdaaa5e9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.920239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fa75c-857f-4950-97a8-5a9121c64a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.921059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fa75c-857f-4950-97a8-5a9121c64a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.921340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fa75c-857f-4950-97a8-5a9121c64a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.921644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fa75c-857f-4950-97a8-5a9121c64a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.922297 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fa75c-857f-4950-97a8-5a9121c64a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:09.938174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fa75c-857f-4950-97a8-5a9121c64a2a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:09.957914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1043/2100] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:09.961154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e64898-c6b2-459b-9f37-d2edaa12e965 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:09.976188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e64898-c6b2-459b-9f37-d2edaa12e965 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:09.981633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5af75-47c3-422d-8007-79cb3c934aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:09.983054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5af75-47c3-422d-8007-79cb3c934aba admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:09.983335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5af75-47c3-422d-8007-79cb3c934aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:09.983610 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5af75-47c3-422d-8007-79cb3c934aba admin admin] PATH_INFO: `/v3/projects/4219587a348646a9a0bead916a08f096/users/50395e9615b64a10bc333660da102b8a/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:09.999856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5af75-47c3-422d-8007-79cb3c934aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4219587a348646a9a0bead916a08f096, user_id=50395e9615b64a10bc333660da102b8a, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:10.002577 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5af75-47c3-422d-8007-79cb3c934aba admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:10.021194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1058/2101] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:09 2022] PUT /identity/v3/projects/4219587a348646a9a0bead916a08f096/users/50395e9615b64a10bc333660da102b8a/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:10.022536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fa75c-857f-4950-97a8-5a9121c64a2a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:10.039982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fa75c-857f-4950-97a8-5a9121c64a2a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:10.063042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:10.079452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5af75-47c3-422d-8007-79cb3c934aba admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:10.081431 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-adc5af75-47c3-422d-8007-79cb3c934aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:10.149612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16587591-010a-48a2-9ad7-12cd4c3b4a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:10.150236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16587591-010a-48a2-9ad7-12cd4c3b4a45 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:10.150841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16587591-010a-48a2-9ad7-12cd4c3b4a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:10.151244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16587591-010a-48a2-9ad7-12cd4c3b4a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:10.151901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16587591-010a-48a2-9ad7-12cd4c3b4a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:10.154700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16587591-010a-48a2-9ad7-12cd4c3b4a45 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:10.166411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1044/2102] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:23:10.170739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16587591-010a-48a2-9ad7-12cd4c3b4a45 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:10.181783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16587591-010a-48a2-9ad7-12cd4c3b4a45 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:10.186883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3a8f6-b91b-4769-b3de-e9e71df00b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:10.187657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3a8f6-b91b-4769-b3de-e9e71df00b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:10.188003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3a8f6-b91b-4769-b3de-e9e71df00b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:10.188434 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3a8f6-b91b-4769-b3de-e9e71df00b63 admin admin] PATH_INFO: `/v3/projects/4219587a348646a9a0bead916a08f096/users/50395e9615b64a10bc333660da102b8a/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:10.190402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3a8f6-b91b-4769-b3de-e9e71df00b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4219587a348646a9a0bead916a08f096, user_id=50395e9615b64a10bc333660da102b8a, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:10.190785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5af75-47c3-422d-8007-79cb3c934aba admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:10.193616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3a8f6-b91b-4769-b3de-e9e71df00b63 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:10.196735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:10.205115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1045/2103] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:10 2022] PUT /identity/v3/projects/4219587a348646a9a0bead916a08f096/users/50395e9615b64a10bc333660da102b8a/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:10.211249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:10.211605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:10.211888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] PATH_INFO: `/v3/projects/973809a885644f4199c8bc707b52a829` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:10.214660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=973809a885644f4199c8bc707b52a829)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:10.217233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:10.217447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94925e03-f62a-4194-a06d-8a764f125864 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:10.217774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94925e03-f62a-4194-a06d-8a764f125864 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:10.218096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94925e03-f62a-4194-a06d-8a764f125864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:10.261495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-94925e03-f62a-4194-a06d-8a764f125864 None None] MFA Rules not processed for user `50395e9615b64a10bc333660da102b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:10.263199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-94925e03-f62a-4194-a06d-8a764f125864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:10.303651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973809a885644f4199c8bc707b52a829', 'request_id': 'req-333853ab-da8c-4e7b-ba6f-7098f21b55ce'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.316084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973809a885644f4199c8bc707b52a829', 'request_id': 'req-333853ab-da8c-4e7b-ba6f-7098f21b55ce'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.332886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973809a885644f4199c8bc707b52a829', 'request_id': 'req-333853ab-da8c-4e7b-ba6f-7098f21b55ce'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.346826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] The token cache is being invalidate because project 973809a885644f4199c8bc707b52a829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:10.347069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.347867 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.349663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1059/2104] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:10 2022] DELETE /identity/v3/projects/973809a885644f4199c8bc707b52a829 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:10.353501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:10.355482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:10.388750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1046/2105] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:23:10.402549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94925e03-f62a-4194-a06d-8a764f125864 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:10.404631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-94925e03-f62a-4194-a06d-8a764f125864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:10.439440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333853ab-da8c-4e7b-ba6f-7098f21b55ce admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:10.444740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:10.445384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:10.445696 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:10.445972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] PATH_INFO: `/v3/users/b32844f8eef84160b6eccfa36b2f45ae` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:10.447139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b32844f8eef84160b6eccfa36b2f45ae)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:10.449091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:10.456212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:10.475109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94925e03-f62a-4194-a06d-8a764f125864 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:10.508989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b32844f8eef84160b6eccfa36b2f45ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.509759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b32844f8eef84160b6eccfa36b2f45ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.519383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b32844f8eef84160b6eccfa36b2f45ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.520154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b32844f8eef84160b6eccfa36b2f45ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.528406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:10.531393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b32844f8eef84160b6eccfa36b2f45ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:10.536121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c64d3e-1024-497a-ac72-8ceddee5805c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:10.536759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64d3e-1024-497a-ac72-8ceddee5805c service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:10.537102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64d3e-1024-497a-ac72-8ceddee5805c service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:10.537527 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64d3e-1024-497a-ac72-8ceddee5805c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:10.539847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-91c64d3e-1024-497a-ac72-8ceddee5805c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:10.559647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1060/2106] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:10 2022] DELETE /identity/v3/users/b32844f8eef84160b6eccfa36b2f45ae => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:10.609073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c64d3e-1024-497a-ac72-8ceddee5805c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:10.610449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c64d3e-1024-497a-ac72-8ceddee5805c service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:10.620181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1047/2107] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:10.905301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:10.907044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:11.075978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1654ab7-c84f-4264-87f3-5828ab522cb3 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:11.086363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:11.087108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:11.087385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:11.087654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] PATH_INFO: `/v3/projects/4d43e7b9c9f2433bae823d9c6d13e012` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:11.088877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d43e7b9c9f2433bae823d9c6d13e012)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:11.090991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:11.230428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d43e7b9c9f2433bae823d9c6d13e012', 'request_id': 'req-4a6a2407-7379-46bc-9eea-14b08c1397e9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.247002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d43e7b9c9f2433bae823d9c6d13e012', 'request_id': 'req-4a6a2407-7379-46bc-9eea-14b08c1397e9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.259469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d43e7b9c9f2433bae823d9c6d13e012', 'request_id': 'req-4a6a2407-7379-46bc-9eea-14b08c1397e9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.276643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] The token cache is being invalidate because project 4d43e7b9c9f2433bae823d9c6d13e012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:11.277059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.277911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.279479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1061/2108] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:10 2022] DELETE /identity/v3/projects/4d43e7b9c9f2433bae823d9c6d13e012 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:11.283645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c64d3e-1024-497a-ac72-8ceddee5805c service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:11.296729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c64d3e-1024-497a-ac72-8ceddee5805c service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:11.420691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:11.421964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:11.422446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:11.422914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] PATH_INFO: `/v3/users/570d0423d43740a096e7276d744430dc` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:11.424474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=570d0423d43740a096e7276d744430dc)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:11.427207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:11.506464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570d0423d43740a096e7276d744430dc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.507275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570d0423d43740a096e7276d744430dc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.516304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570d0423d43740a096e7276d744430dc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.530386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570d0423d43740a096e7276d744430dc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.531129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-a59131d1-27c1-418a-9b35-dff5024c5b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570d0423d43740a096e7276d744430dc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:11.554883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1048/2109] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:11 2022] DELETE /identity/v3/users/570d0423d43740a096e7276d744430dc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:12.478008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:12.485403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:12.524785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a2407-7379-46bc-9eea-14b08c1397e9 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:12.541468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:12.655381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:12.657090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:12.657467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:12.658037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] PATH_INFO: `/v3/projects/20bd2b69c9414e678ddf5518fd175f80` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:12.659327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=20bd2b69c9414e678ddf5518fd175f80)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:12.661574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:12.764368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20bd2b69c9414e678ddf5518fd175f80', 'request_id': 'req-8b2289a3-14f1-4f47-8665-6f40fe14e84f'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:12.778134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20bd2b69c9414e678ddf5518fd175f80', 'request_id': 'req-8b2289a3-14f1-4f47-8665-6f40fe14e84f'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:12.801560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20bd2b69c9414e678ddf5518fd175f80', 'request_id': 'req-8b2289a3-14f1-4f47-8665-6f40fe14e84f'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:12.817418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] The token cache is being invalidate because project 20bd2b69c9414e678ddf5518fd175f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:12.818281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:12.819161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8b2289a3-14f1-4f47-8665-6f40fe14e84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:12.825575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1062/2110] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:12 2022] DELETE /identity/v3/projects/20bd2b69c9414e678ddf5518fd175f80 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:12.826468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eb497-97be-4579-807d-acc2c5dddef0 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:12.826916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eb497-97be-4579-807d-acc2c5dddef0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:12.827018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eb497-97be-4579-807d-acc2c5dddef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:12.850968 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-fc5eb497-97be-4579-807d-acc2c5dddef0 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:12.852259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fc5eb497-97be-4579-807d-acc2c5dddef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:12.879105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:12.936365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1049/2111] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:12.945594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f6476-98f7-43a0-9af0-70c09b24d9dc None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:12.945880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f6476-98f7-43a0-9af0-70c09b24d9dc None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:12.946151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f6476-98f7-43a0-9af0-70c09b24d9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:12.967823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a92f6476-98f7-43a0-9af0-70c09b24d9dc None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:12.969221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a92f6476-98f7-43a0-9af0-70c09b24d9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:12.978964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1063/2112] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:12.983813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eb497-97be-4579-807d-acc2c5dddef0 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:12.995369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eb497-97be-4579-807d-acc2c5dddef0 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:13.010930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f12c0-d933-4cc6-98aa-e4288f8df731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:13.011643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f12c0-d933-4cc6-98aa-e4288f8df731 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:13.011977 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f12c0-d933-4cc6-98aa-e4288f8df731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:13.012325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f12c0-d933-4cc6-98aa-e4288f8df731 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:13.013037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f12c0-d933-4cc6-98aa-e4288f8df731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:23:13.013340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f12c0-d933-4cc6-98aa-e4288f8df731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:13.015033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f12c0-d933-4cc6-98aa-e4288f8df731 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:13.027136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1050/2113] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:23:13.032130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f6476-98f7-43a0-9af0-70c09b24d9dc None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:13.041377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f6476-98f7-43a0-9af0-70c09b24d9dc None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:13.047243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7654804-84a2-4ecc-9715-2ea5395f35cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:13.047844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7654804-84a2-4ecc-9715-2ea5395f35cb admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:13.048117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7654804-84a2-4ecc-9715-2ea5395f35cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:13.048385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7654804-84a2-4ecc-9715-2ea5395f35cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:13.049031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7654804-84a2-4ecc-9715-2ea5395f35cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:13.051063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7654804-84a2-4ecc-9715-2ea5395f35cb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:13.079349 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1064/2114] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:13 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:13.083062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f12c0-d933-4cc6-98aa-e4288f8df731 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:13.094900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f12c0-d933-4cc6-98aa-e4288f8df731 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:13.157796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949962a-3b70-4971-becb-69188fabd107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:13.158402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949962a-3b70-4971-becb-69188fabd107 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:13.158678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949962a-3b70-4971-becb-69188fabd107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:13.158953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949962a-3b70-4971-becb-69188fabd107 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:13.159587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949962a-3b70-4971-becb-69188fabd107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:13.161496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949962a-3b70-4971-becb-69188fabd107 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:13.182473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1051/2115] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:13 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:13.186609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7654804-84a2-4ecc-9715-2ea5395f35cb admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:13.196954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7654804-84a2-4ecc-9715-2ea5395f35cb admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:13.201543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0080d-f403-42ba-b616-75f151792250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:13.202143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0080d-f403-42ba-b616-75f151792250 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:13.202418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0080d-f403-42ba-b616-75f151792250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:13.202685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0080d-f403-42ba-b616-75f151792250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:13.203328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0080d-f403-42ba-b616-75f151792250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:13.204936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0080d-f403-42ba-b616-75f151792250 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:13.214657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1065/2116] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:13.219006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949962a-3b70-4971-becb-69188fabd107 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:13.228700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949962a-3b70-4971-becb-69188fabd107 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:13.233873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb1b27-4ccf-4986-b538-f1b3d0331272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:13.234457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb1b27-4ccf-4986-b538-f1b3d0331272 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:13.234732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb1b27-4ccf-4986-b538-f1b3d0331272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:13.235000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb1b27-4ccf-4986-b538-f1b3d0331272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:13.235627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb1b27-4ccf-4986-b538-f1b3d0331272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:13.237193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb1b27-4ccf-4986-b538-f1b3d0331272 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:13.248581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1052/2117] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:13.252555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0080d-f403-42ba-b616-75f151792250 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:13.262413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0080d-f403-42ba-b616-75f151792250 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:13.267806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9067505-1c3c-4905-8f8c-a190d204e336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:13.268564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9067505-1c3c-4905-8f8c-a190d204e336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:13.268849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9067505-1c3c-4905-8f8c-a190d204e336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:13.269123 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9067505-1c3c-4905-8f8c-a190d204e336 admin admin] PATH_INFO: `/v3/projects/3bcb53a0f7a44ffe8e43dce9fdb11142/users/cd8bca48472641af9123a5caa9817b0f/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:13.283953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9067505-1c3c-4905-8f8c-a190d204e336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bcb53a0f7a44ffe8e43dce9fdb11142, user_id=cd8bca48472641af9123a5caa9817b0f, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:13.286716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9067505-1c3c-4905-8f8c-a190d204e336 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:13.300295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1066/2118] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:13 2022] PUT /identity/v3/projects/3bcb53a0f7a44ffe8e43dce9fdb11142/users/cd8bca48472641af9123a5caa9817b0f/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:13.310609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed0a2b-3a3e-46f1-a5f4-e7f1129d0cec None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:13.310895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed0a2b-3a3e-46f1-a5f4-e7f1129d0cec None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:13.311187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed0a2b-3a3e-46f1-a5f4-e7f1129d0cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:13.339325 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-7eed0a2b-3a3e-46f1-a5f4-e7f1129d0cec None None] MFA Rules not processed for user `cd8bca48472641af9123a5caa9817b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:13.340579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7eed0a2b-3a3e-46f1-a5f4-e7f1129d0cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:13.402948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1053/2119] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:13.414551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9067505-1c3c-4905-8f8c-a190d204e336 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:13.416219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b9067505-1c3c-4905-8f8c-a190d204e336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:13.461519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:13.475496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9067505-1c3c-4905-8f8c-a190d204e336 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:13.479345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54026cf2-d291-4c25-86fe-3b04d46a76e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:13.479950 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54026cf2-d291-4c25-86fe-3b04d46a76e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:13.480310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54026cf2-d291-4c25-86fe-3b04d46a76e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:13.480574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54026cf2-d291-4c25-86fe-3b04d46a76e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:13.488865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54026cf2-d291-4c25-86fe-3b04d46a76e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:13.490328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54026cf2-d291-4c25-86fe-3b04d46a76e7 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:13.499150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1067/2120] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:16.805073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed0a2b-3a3e-46f1-a5f4-e7f1129d0cec None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:16.806513 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7eed0a2b-3a3e-46f1-a5f4-e7f1129d0cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:16.818405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed0a2b-3a3e-46f1-a5f4-e7f1129d0cec None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:16.821898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ad3ff-17fc-4f6e-8ae9-9a6b45a557a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:16.822490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ad3ff-17fc-4f6e-8ae9-9a6b45a557a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:16.822764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ad3ff-17fc-4f6e-8ae9-9a6b45a557a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:16.823030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ad3ff-17fc-4f6e-8ae9-9a6b45a557a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:16.824683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c94ad3ff-17fc-4f6e-8ae9-9a6b45a557a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:16.836902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:16.889094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ad3ff-17fc-4f6e-8ae9-9a6b45a557a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:16.890555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ad3ff-17fc-4f6e-8ae9-9a6b45a557a0 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:16.900406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1054/2121] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:23.495098 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=7 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:26.895761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=23 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:31.942572 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228ad08-f673-4bf4-850b-298eb828f488 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:31.943024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228ad08-f673-4bf4-850b-298eb828f488 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:31.943187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228ad08-f673-4bf4-850b-298eb828f488 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:31.944955 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1068/2122] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:23:31.954869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c797b-4af5-48a9-bfd6-0d017c1cc021 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:31.955707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c797b-4af5-48a9-bfd6-0d017c1cc021 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:31.956036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c797b-4af5-48a9-bfd6-0d017c1cc021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:31.977890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c31c797b-4af5-48a9-bfd6-0d017c1cc021 None None] MFA Rules not processed for user `5181c4b31ac240469aff7a0aee35eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:31.979240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c31c797b-4af5-48a9-bfd6-0d017c1cc021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:32.045647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228ad08-f673-4bf4-850b-298eb828f488 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:32.047365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b228ad08-f673-4bf4-850b-298eb828f488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:32.058019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1055/2123] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:23:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:23:32.069417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c797b-4af5-48a9-bfd6-0d017c1cc021 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:32.070941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c31c797b-4af5-48a9-bfd6-0d017c1cc021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:32.083915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c797b-4af5-48a9-bfd6-0d017c1cc021 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:32.088784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba77b4-963f-430b-8f92-1f6bb15ca49d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:32.089380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba77b4-963f-430b-8f92-1f6bb15ca49d service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.089701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba77b4-963f-430b-8f92-1f6bb15ca49d service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.089969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba77b4-963f-430b-8f92-1f6bb15ca49d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.099049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba77b4-963f-430b-8f92-1f6bb15ca49d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:32.100993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba77b4-963f-430b-8f92-1f6bb15ca49d service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:32.111856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1056/2124] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:23:32.116825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228ad08-f673-4bf4-850b-298eb828f488 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:32.121477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:32.122309 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.122561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.122867 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] PATH_INFO: `/v3/users/a553cd4c2e154ee1979812ae575e6fb2` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.124114 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a553cd4c2e154ee1979812ae575e6fb2)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:32.126243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:32.183442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a553cd4c2e154ee1979812ae575e6fb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.184107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a553cd4c2e154ee1979812ae575e6fb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.194489 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a553cd4c2e154ee1979812ae575e6fb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.195181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a553cd4c2e154ee1979812ae575e6fb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.203624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-77648d9f-62e4-4cd2-8544-71868e9da7f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a553cd4c2e154ee1979812ae575e6fb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.231641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1069/2125] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:32 2022] DELETE /identity/v3/users/a553cd4c2e154ee1979812ae575e6fb2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:32.428414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba77b4-963f-430b-8f92-1f6bb15ca49d service placement] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:32.449538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba77b4-963f-430b-8f92-1f6bb15ca49d service placement] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:32.504679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:32.505290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.505601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.505871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] PATH_INFO: `/v3/projects/38145c704ac548abb6747e073892ff72` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.507062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38145c704ac548abb6747e073892ff72)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:32.508994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:32.553714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38145c704ac548abb6747e073892ff72', 'request_id': 'req-7e54f2b5-2faf-424f-b572-6835b7f45967'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.562643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38145c704ac548abb6747e073892ff72', 'request_id': 'req-7e54f2b5-2faf-424f-b572-6835b7f45967'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.569111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38145c704ac548abb6747e073892ff72', 'request_id': 'req-7e54f2b5-2faf-424f-b572-6835b7f45967'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.580581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] The token cache is being invalidate because project 38145c704ac548abb6747e073892ff72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:32.580874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.581577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7e54f2b5-2faf-424f-b572-6835b7f45967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:32.583107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1057/2126] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:32 2022] DELETE /identity/v3/projects/38145c704ac548abb6747e073892ff72 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:32.587949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e70135-64c4-4cd7-a121-db27e8f24e3e None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.588236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e70135-64c4-4cd7-a121-db27e8f24e3e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.588505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e70135-64c4-4cd7-a121-db27e8f24e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.607925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-42e70135-64c4-4cd7-a121-db27e8f24e3e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:32.609047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-42e70135-64c4-4cd7-a121-db27e8f24e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:32.671335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1070/2127] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:32.679505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd5cc-6a29-4550-9731-7899de0ca50d None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.679794 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd5cc-6a29-4550-9731-7899de0ca50d None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.680065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd5cc-6a29-4550-9731-7899de0ca50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.701174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ee2cd5cc-6a29-4550-9731-7899de0ca50d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:32.702405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ee2cd5cc-6a29-4550-9731-7899de0ca50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:32.710141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1058/2128] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:32.714276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e70135-64c4-4cd7-a121-db27e8f24e3e None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:32.723991 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e70135-64c4-4cd7-a121-db27e8f24e3e None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:32.728527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92b7b8-c896-49f2-b948-71b99173869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:32.729117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92b7b8-c896-49f2-b948-71b99173869e admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.729446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92b7b8-c896-49f2-b948-71b99173869e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.729760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92b7b8-c896-49f2-b948-71b99173869e admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.730396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92b7b8-c896-49f2-b948-71b99173869e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:23:32.730723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92b7b8-c896-49f2-b948-71b99173869e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:32.732311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92b7b8-c896-49f2-b948-71b99173869e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:32.743567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1071/2129] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:23:32.748120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cd5cc-6a29-4550-9731-7899de0ca50d None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:32.757270 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cd5cc-6a29-4550-9731-7899de0ca50d None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:32.761848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64be93-ecd2-4299-b2e7-c1b5e7c98586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:32.763084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64be93-ecd2-4299-b2e7-c1b5e7c98586 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.763361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64be93-ecd2-4299-b2e7-c1b5e7c98586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.763629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64be93-ecd2-4299-b2e7-c1b5e7c98586 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.764255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64be93-ecd2-4299-b2e7-c1b5e7c98586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:32.766170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64be93-ecd2-4299-b2e7-c1b5e7c98586 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:32.770555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:32.794591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1059/2130] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:32 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:32.798645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92b7b8-c896-49f2-b948-71b99173869e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:32.808559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92b7b8-c896-49f2-b948-71b99173869e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:32.812691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:32.866243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269555d3-574f-4ad8-8bd0-ef681f86edbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:32.866767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269555d3-574f-4ad8-8bd0-ef681f86edbc admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.867042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269555d3-574f-4ad8-8bd0-ef681f86edbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.867315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269555d3-574f-4ad8-8bd0-ef681f86edbc admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.867952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269555d3-574f-4ad8-8bd0-ef681f86edbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:32.870052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269555d3-574f-4ad8-8bd0-ef681f86edbc admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:32.892207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1072/2131] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:32 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:32.896236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64be93-ecd2-4299-b2e7-c1b5e7c98586 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:32.906459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64be93-ecd2-4299-b2e7-c1b5e7c98586 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:32.910955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55194f34-444f-484d-ab6e-98ad6e8bfd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:32.911547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55194f34-444f-484d-ab6e-98ad6e8bfd7f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.911822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55194f34-444f-484d-ab6e-98ad6e8bfd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.912092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55194f34-444f-484d-ab6e-98ad6e8bfd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.912730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55194f34-444f-484d-ab6e-98ad6e8bfd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:32.914410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55194f34-444f-484d-ab6e-98ad6e8bfd7f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:32.924817 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1060/2132] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:32.928603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269555d3-574f-4ad8-8bd0-ef681f86edbc admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:32.939086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269555d3-574f-4ad8-8bd0-ef681f86edbc admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:32.943927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bfbc5-9f7a-4ab4-9741-a8a15b84fe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:32.944523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bfbc5-9f7a-4ab4-9741-a8a15b84fe1c admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.944775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bfbc5-9f7a-4ab4-9741-a8a15b84fe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.945434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bfbc5-9f7a-4ab4-9741-a8a15b84fe1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.945761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bfbc5-9f7a-4ab4-9741-a8a15b84fe1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:32.947340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bfbc5-9f7a-4ab4-9741-a8a15b84fe1c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:32.959254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1073/2133] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:32.963280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55194f34-444f-484d-ab6e-98ad6e8bfd7f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:32.973126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55194f34-444f-484d-ab6e-98ad6e8bfd7f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:32.977697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1bca8-81d5-4c41-b233-67e125cbb21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:32.978334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1bca8-81d5-4c41-b233-67e125cbb21a admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:32.978609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1bca8-81d5-4c41-b233-67e125cbb21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:32.978876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1bca8-81d5-4c41-b233-67e125cbb21a admin admin] PATH_INFO: `/v3/projects/05398ef99be34b57b61052a0f07f57f8/users/10d2803e109849fda2828f66f51b8a0c/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:32.993279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1bca8-81d5-4c41-b233-67e125cbb21a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05398ef99be34b57b61052a0f07f57f8, user_id=10d2803e109849fda2828f66f51b8a0c, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:32.995931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb1bca8-81d5-4c41-b233-67e125cbb21a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:33.009328 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1061/2134] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:32 2022] PUT /identity/v3/projects/05398ef99be34b57b61052a0f07f57f8/users/10d2803e109849fda2828f66f51b8a0c/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:33.014954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197efd6-e5ca-4985-af06-0da82c6c3e13 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:33.015292 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197efd6-e5ca-4985-af06-0da82c6c3e13 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:33.015599 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197efd6-e5ca-4985-af06-0da82c6c3e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:33.034462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-2197efd6-e5ca-4985-af06-0da82c6c3e13 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:33.035650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2197efd6-e5ca-4985-af06-0da82c6c3e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:33.095448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1074/2135] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:33.107293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1bca8-81d5-4c41-b233-67e125cbb21a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:33.108695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-acb1bca8-81d5-4c41-b233-67e125cbb21a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:33.180025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb1bca8-81d5-4c41-b233-67e125cbb21a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:33.185048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e90fa-c175-4da9-8301-082d035852be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:33.185048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e90fa-c175-4da9-8301-082d035852be service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:33.185048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e90fa-c175-4da9-8301-082d035852be service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:33.185326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e90fa-c175-4da9-8301-082d035852be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:33.200633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e90fa-c175-4da9-8301-082d035852be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:33.202149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e90fa-c175-4da9-8301-082d035852be service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:33.212677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1062/2136] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:37.804811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200229-6dbf-4086-a7b1-08eb9706f101 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:37.805286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200229-6dbf-4086-a7b1-08eb9706f101 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:37.805471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4200229-6dbf-4086-a7b1-08eb9706f101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:37.842974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b4200229-6dbf-4086-a7b1-08eb9706f101 None None] MFA Rules not processed for user `10d2803e109849fda2828f66f51b8a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:37.844124 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b4200229-6dbf-4086-a7b1-08eb9706f101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:37.925766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1075/2137] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:37.940373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e90fa-c175-4da9-8301-082d035852be service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:37.942117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e85e90fa-c175-4da9-8301-082d035852be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:38.029570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e90fa-c175-4da9-8301-082d035852be service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:38.035060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9a917-6097-4b55-b580-f8e703c660e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:38.035675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9a917-6097-4b55-b580-f8e703c660e6 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:38.035953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9a917-6097-4b55-b580-f8e703c660e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:38.036226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9a917-6097-4b55-b580-f8e703c660e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:38.045124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9a917-6097-4b55-b580-f8e703c660e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:38.046737 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9a917-6097-4b55-b580-f8e703c660e6 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:38.056844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1063/2138] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:39.855364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8e5a0-5cf9-4757-8bb6-050ae3714de0 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:39.855880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8e5a0-5cf9-4757-8bb6-050ae3714de0 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:39.856242 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8e5a0-5cf9-4757-8bb6-050ae3714de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:39.878422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-3cc8e5a0-5cf9-4757-8bb6-050ae3714de0 None None] MFA Rules not processed for user `10d2803e109849fda2828f66f51b8a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:39.880391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3cc8e5a0-5cf9-4757-8bb6-050ae3714de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:39.890577 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1076/2139] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:23:40.547090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9a917-6097-4b55-b580-f8e703c660e6 service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:40.549218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-50e9a917-6097-4b55-b580-f8e703c660e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:40.586040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9a917-6097-4b55-b580-f8e703c660e6 service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:40.598471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:40.600444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:40.600708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:40.601499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] PATH_INFO: `/v3/users/cd8bca48472641af9123a5caa9817b0f` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:40.605030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd8bca48472641af9123a5caa9817b0f)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:40.625638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:40.724059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:40.757085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd8bca48472641af9123a5caa9817b0f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:40.758023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd8bca48472641af9123a5caa9817b0f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:40.785155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd8bca48472641af9123a5caa9817b0f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:40.826593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd8bca48472641af9123a5caa9817b0f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:40.828127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd8bca48472641af9123a5caa9817b0f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:40.892575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1064/2140] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:40 2022] DELETE /identity/v3/users/cd8bca48472641af9123a5caa9817b0f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:40.897223 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be673fcd-0916-4243-93e1-c3e7555b1ea5 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:40.898293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be673fcd-0916-4243-93e1-c3e7555b1ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:40.898703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be673fcd-0916-4243-93e1-c3e7555b1ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:40.964138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-be673fcd-0916-4243-93e1-c3e7555b1ea5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:40.967454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-be673fcd-0916-4243-93e1-c3e7555b1ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:40.974293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:41.115248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1077/2141] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:41.125719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:41.128761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:41.195085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58825383-30f7-4742-9ebd-2e997eae6b31 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:41.199339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d03906-0436-442a-ad3e-dbf0d2b0453c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:41.200002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d03906-0436-442a-ad3e-dbf0d2b0453c service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:41.200583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d03906-0436-442a-ad3e-dbf0d2b0453c service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:41.200583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d03906-0436-442a-ad3e-dbf0d2b0453c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:41.212045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d03906-0436-442a-ad3e-dbf0d2b0453c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:41.213598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d03906-0436-442a-ad3e-dbf0d2b0453c service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:41.225690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1065/2142] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:41.510975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be673fcd-0916-4243-93e1-c3e7555b1ea5 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:41.513162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-be673fcd-0916-4243-93e1-c3e7555b1ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:41.536293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be673fcd-0916-4243-93e1-c3e7555b1ea5 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:41.541164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:41.542573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:41.542988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:41.543155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] PATH_INFO: `/v3/projects/3bcb53a0f7a44ffe8e43dce9fdb11142` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:41.545804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bcb53a0f7a44ffe8e43dce9fdb11142)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:41.547842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:41.598062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bcb53a0f7a44ffe8e43dce9fdb11142', 'request_id': 'req-72532857-710f-41a9-a275-93d6bf73de1e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:41.606897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bcb53a0f7a44ffe8e43dce9fdb11142', 'request_id': 'req-72532857-710f-41a9-a275-93d6bf73de1e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:41.613467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bcb53a0f7a44ffe8e43dce9fdb11142', 'request_id': 'req-72532857-710f-41a9-a275-93d6bf73de1e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:41.624475 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] The token cache is being invalidate because project 3bcb53a0f7a44ffe8e43dce9fdb11142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:41.624647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:41.626071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:41.628132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1078/2143] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:41 2022] DELETE /identity/v3/projects/3bcb53a0f7a44ffe8e43dce9fdb11142 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:48.810096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86deb53-4120-4a13-881d-7afd273ea561 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:48.810618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86deb53-4120-4a13-881d-7afd273ea561 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:48.810618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86deb53-4120-4a13-881d-7afd273ea561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:48.837184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a86deb53-4120-4a13-881d-7afd273ea561 None None] MFA Rules not processed for user `1b8c41cfa6854a528f43e82e0fb26220`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:48.838407 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a86deb53-4120-4a13-881d-7afd273ea561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:48.911084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1066/2144] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:49.083631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:49.085096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:49.163345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532857-710f-41a9-a275-93d6bf73de1e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:49.168377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:49.169036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:49.169320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:49.169630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] PATH_INFO: `/v3/users/53d80bf4a8724d30adfb9f0177b0ed1a` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:49.170812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d80bf4a8724d30adfb9f0177b0ed1a)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:49.172783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:49.225565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:49.238601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d80bf4a8724d30adfb9f0177b0ed1a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:49.239608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d80bf4a8724d30adfb9f0177b0ed1a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:49.249631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d80bf4a8724d30adfb9f0177b0ed1a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:49.250353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d80bf4a8724d30adfb9f0177b0ed1a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:49.265172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d80bf4a8724d30adfb9f0177b0ed1a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:49.295168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1079/2145] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:49 2022] DELETE /identity/v3/users/53d80bf4a8724d30adfb9f0177b0ed1a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:49.296747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4749472-9899-4f22-beca-adb8fa0e1e72 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:49.297035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4749472-9899-4f22-beca-adb8fa0e1e72 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:49.297311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4749472-9899-4f22-beca-adb8fa0e1e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:49.333094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e4749472-9899-4f22-beca-adb8fa0e1e72 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:49.335462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e4749472-9899-4f22-beca-adb8fa0e1e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:49.404849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:49.416458 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1067/2146] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:49.425926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:49.427630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:49.507194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173191fa-5d67-4904-b641-e6924fbd75e2 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:49.510907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bbd6a-b479-4e21-b849-83fba56e74cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:49.511511 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bbd6a-b479-4e21-b849-83fba56e74cf service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:49.512367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bbd6a-b479-4e21-b849-83fba56e74cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:49.512636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bbd6a-b479-4e21-b849-83fba56e74cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:49.523042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bbd6a-b479-4e21-b849-83fba56e74cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:49.524481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bbd6a-b479-4e21-b849-83fba56e74cf service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:49.535632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1080/2147] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:49.870180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4749472-9899-4f22-beca-adb8fa0e1e72 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:49.873285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e4749472-9899-4f22-beca-adb8fa0e1e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:49.890934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4749472-9899-4f22-beca-adb8fa0e1e72 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:49.912660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:49.913404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:49.913778 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:49.914118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] PATH_INFO: `/v3/projects/afa4428e70ac431882b3012118d5c049` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:49.915337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa4428e70ac431882b3012118d5c049)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:49.918782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:49.989137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa4428e70ac431882b3012118d5c049', 'request_id': 'req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:49.999324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa4428e70ac431882b3012118d5c049', 'request_id': 'req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.006587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa4428e70ac431882b3012118d5c049', 'request_id': 'req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.022598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] The token cache is being invalidate because project afa4428e70ac431882b3012118d5c049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:50.022901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.023734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.025359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1068/2148] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:49 2022] DELETE /identity/v3/projects/afa4428e70ac431882b3012118d5c049 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:50.029438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bbd6a-b479-4e21-b849-83fba56e74cf service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:50.039515 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bbd6a-b479-4e21-b849-83fba56e74cf service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:50.110210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:50.110956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:50.111296 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:50.111630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] PATH_INFO: `/v3/users/50395e9615b64a10bc333660da102b8a` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:50.112838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=50395e9615b64a10bc333660da102b8a)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:50.114871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:50.162072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50395e9615b64a10bc333660da102b8a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.162903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50395e9615b64a10bc333660da102b8a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.172794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50395e9615b64a10bc333660da102b8a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.173725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50395e9615b64a10bc333660da102b8a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.175763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:50.182884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f981db28-3ac3-4520-836b-b372996eb91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50395e9615b64a10bc333660da102b8a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.202734 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1081/2149] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:23:50 2022] DELETE /identity/v3/users/50395e9615b64a10bc333660da102b8a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:50.473016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:50.474549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:50.490792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212cdf0-bff5-4ec1-95ab-a2c3e1f2599b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:50.553866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:50.554421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:50.554696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:50.554965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] PATH_INFO: `/v3/projects/4219587a348646a9a0bead916a08f096` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:50.556173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4219587a348646a9a0bead916a08f096)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:50.558175 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:50.611359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4219587a348646a9a0bead916a08f096', 'request_id': 'req-d2269428-1750-456e-a119-0c7e66fef75b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.619381 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4219587a348646a9a0bead916a08f096', 'request_id': 'req-d2269428-1750-456e-a119-0c7e66fef75b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.626356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4219587a348646a9a0bead916a08f096', 'request_id': 'req-d2269428-1750-456e-a119-0c7e66fef75b'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.638093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] The token cache is being invalidate because project 4219587a348646a9a0bead916a08f096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:23:50.638382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.639042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2269428-1750-456e-a119-0c7e66fef75b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:23:50.640735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1069/2150] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:23:50 2022] DELETE /identity/v3/projects/4219587a348646a9a0bead916a08f096 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:50.645643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e3549-f3c7-4fa2-b43e-10849c148657 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:50.646151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e3549-f3c7-4fa2-b43e-10849c148657 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:50.646421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e3549-f3c7-4fa2-b43e-10849c148657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:50.667496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d38e3549-f3c7-4fa2-b43e-10849c148657 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:50.669432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d38e3549-f3c7-4fa2-b43e-10849c148657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:50.731041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1082/2151] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:23:50.739940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a97e07-6d86-4e06-9c4b-ebb6c08cc5e2 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:50.740217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a97e07-6d86-4e06-9c4b-ebb6c08cc5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:50.740493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a97e07-6d86-4e06-9c4b-ebb6c08cc5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:50.762639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f2a97e07-6d86-4e06-9c4b-ebb6c08cc5e2 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:50.763952 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f2a97e07-6d86-4e06-9c4b-ebb6c08cc5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:50.779587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1070/2152] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:50.781369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e3549-f3c7-4fa2-b43e-10849c148657 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:50.794194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e3549-f3c7-4fa2-b43e-10849c148657 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:50.799946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da4ba7-80b9-4c1d-9a9f-12b3a290d9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:50.800579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da4ba7-80b9-4c1d-9a9f-12b3a290d9eb admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:50.800994 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da4ba7-80b9-4c1d-9a9f-12b3a290d9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:50.801270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da4ba7-80b9-4c1d-9a9f-12b3a290d9eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:50.805864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da4ba7-80b9-4c1d-9a9f-12b3a290d9eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:23:50.805864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da4ba7-80b9-4c1d-9a9f-12b3a290d9eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:50.805864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da4ba7-80b9-4c1d-9a9f-12b3a290d9eb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:50.824865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1083/2153] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:23:50.826821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a97e07-6d86-4e06-9c4b-ebb6c08cc5e2 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:50.839345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:50.858397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a97e07-6d86-4e06-9c4b-ebb6c08cc5e2 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:50.898502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38bdf1-d9bb-4377-abbe-2756135b2a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:50.899491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38bdf1-d9bb-4377-abbe-2756135b2a34 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:50.901554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38bdf1-d9bb-4377-abbe-2756135b2a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:50.902000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38bdf1-d9bb-4377-abbe-2756135b2a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:50.903334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38bdf1-d9bb-4377-abbe-2756135b2a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:50.905253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38bdf1-d9bb-4377-abbe-2756135b2a34 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:50.965599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1071/2154] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:50 2022] POST /identity/v3/projects => generated 407 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:50.969708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da4ba7-80b9-4c1d-9a9f-12b3a290d9eb admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:50.988660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da4ba7-80b9-4c1d-9a9f-12b3a290d9eb admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.062266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe0f7-a259-49c4-a922-54550cf1ac1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.062895 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe0f7-a259-49c4-a922-54550cf1ac1c admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.063173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe0f7-a259-49c4-a922-54550cf1ac1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.063447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe0f7-a259-49c4-a922-54550cf1ac1c admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.064094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbe0f7-a259-49c4-a922-54550cf1ac1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.066179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbe0f7-a259-49c4-a922-54550cf1ac1c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.094437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1084/2155] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:50 2022] POST /identity/v3/users => generated 304 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:51.098944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38bdf1-d9bb-4377-abbe-2756135b2a34 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.112172 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38bdf1-d9bb-4377-abbe-2756135b2a34 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.116771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1595e01-ff75-4e2e-9ee5-76023068db38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.118160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1595e01-ff75-4e2e-9ee5-76023068db38 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.118160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1595e01-ff75-4e2e-9ee5-76023068db38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.118160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1595e01-ff75-4e2e-9ee5-76023068db38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.118564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1595e01-ff75-4e2e-9ee5-76023068db38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.127485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1595e01-ff75-4e2e-9ee5-76023068db38 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.149693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1072/2156] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:51.151330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe0f7-a259-49c4-a922-54550cf1ac1c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.165822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe0f7-a259-49c4-a922-54550cf1ac1c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.170831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b468602-c322-4360-b03e-e26e2eca5240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.171495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b468602-c322-4360-b03e-e26e2eca5240 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.171770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b468602-c322-4360-b03e-e26e2eca5240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.172049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b468602-c322-4360-b03e-e26e2eca5240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.172720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b468602-c322-4360-b03e-e26e2eca5240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.174348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b468602-c322-4360-b03e-e26e2eca5240 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.186799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1085/2157] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:23:51.190961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1595e01-ff75-4e2e-9ee5-76023068db38 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.205059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1595e01-ff75-4e2e-9ee5-76023068db38 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.211146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738516e8-41d7-46c8-9709-a6f1c4f4f6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.212205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738516e8-41d7-46c8-9709-a6f1c4f4f6f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.212673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738516e8-41d7-46c8-9709-a6f1c4f4f6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.213134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738516e8-41d7-46c8-9709-a6f1c4f4f6f8 admin admin] PATH_INFO: `/v3/projects/5a1b4f43e75d4169a7ef558fc2debce9/users/3f0f9dd9d06d464d93df04b57e94abcf/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.230766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738516e8-41d7-46c8-9709-a6f1c4f4f6f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1b4f43e75d4169a7ef558fc2debce9, user_id=3f0f9dd9d06d464d93df04b57e94abcf, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.236942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738516e8-41d7-46c8-9709-a6f1c4f4f6f8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.256760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1073/2158] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:51 2022] PUT /identity/v3/projects/5a1b4f43e75d4169a7ef558fc2debce9/users/3f0f9dd9d06d464d93df04b57e94abcf/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:51.269240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4acf8-37ff-49b9-8340-e6dfd4c87f1e None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.270926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4acf8-37ff-49b9-8340-e6dfd4c87f1e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.271200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4acf8-37ff-49b9-8340-e6dfd4c87f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.311101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-00f4acf8-37ff-49b9-8340-e6dfd4c87f1e None None] MFA Rules not processed for user `3f0f9dd9d06d464d93df04b57e94abcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:51.312366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-00f4acf8-37ff-49b9-8340-e6dfd4c87f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:51.347659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:51.389804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1086/2159] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:51 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:51.393131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738516e8-41d7-46c8-9709-a6f1c4f4f6f8 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.458787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738516e8-41d7-46c8-9709-a6f1c4f4f6f8 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.463492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c28f6c-8e97-498c-9add-b9b858ddef6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.464099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c28f6c-8e97-498c-9add-b9b858ddef6d admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.464377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c28f6c-8e97-498c-9add-b9b858ddef6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.464646 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c28f6c-8e97-498c-9add-b9b858ddef6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.465316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c28f6c-8e97-498c-9add-b9b858ddef6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.467247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c28f6c-8e97-498c-9add-b9b858ddef6d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.494226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1074/2160] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:23:51 2022] POST /identity/v3/projects => generated 409 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:51.499381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4acf8-37ff-49b9-8340-e6dfd4c87f1e None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.510130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4acf8-37ff-49b9-8340-e6dfd4c87f1e None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.567121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4a062-6803-43ca-b2e4-d02de3f74125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.567632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4a062-6803-43ca-b2e4-d02de3f74125 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.567907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4a062-6803-43ca-b2e4-d02de3f74125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.568174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4a062-6803-43ca-b2e4-d02de3f74125 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.568798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4a062-6803-43ca-b2e4-d02de3f74125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.570802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4a062-6803-43ca-b2e4-d02de3f74125 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.592211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1087/2161] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:23:51 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:23:51.596284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c28f6c-8e97-498c-9add-b9b858ddef6d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.607406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c28f6c-8e97-498c-9add-b9b858ddef6d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.611945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9eff3-1d9c-469f-80c2-88ff79b8aecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.612536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9eff3-1d9c-469f-80c2-88ff79b8aecf admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.612809 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9eff3-1d9c-469f-80c2-88ff79b8aecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.613076 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9eff3-1d9c-469f-80c2-88ff79b8aecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.613742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9eff3-1d9c-469f-80c2-88ff79b8aecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.615348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9eff3-1d9c-469f-80c2-88ff79b8aecf admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.626916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1075/2162] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:51.631229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4a062-6803-43ca-b2e4-d02de3f74125 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.641348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4a062-6803-43ca-b2e4-d02de3f74125 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.646125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cbbb6-5b1a-4084-aa41-34680b95ce3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.646778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cbbb6-5b1a-4084-aa41-34680b95ce3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.647055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cbbb6-5b1a-4084-aa41-34680b95ce3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.647327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cbbb6-5b1a-4084-aa41-34680b95ce3b admin admin] PATH_INFO: `/v3/projects/6121c8241c37402c88ec43864ea3c7b4/users/399186c0e20c4b51a6be3ef433eec005/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.662423 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cbbb6-5b1a-4084-aa41-34680b95ce3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6121c8241c37402c88ec43864ea3c7b4, user_id=399186c0e20c4b51a6be3ef433eec005, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.665092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cbbb6-5b1a-4084-aa41-34680b95ce3b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.678516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1088/2163] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:51 2022] PUT /identity/v3/projects/6121c8241c37402c88ec43864ea3c7b4/users/399186c0e20c4b51a6be3ef433eec005/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:23:51.682566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9eff3-1d9c-469f-80c2-88ff79b8aecf admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.693574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9eff3-1d9c-469f-80c2-88ff79b8aecf admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.754813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520808ce-da4f-41e9-bef0-2051914c1c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.755550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520808ce-da4f-41e9-bef0-2051914c1c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.755891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520808ce-da4f-41e9-bef0-2051914c1c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.756228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520808ce-da4f-41e9-bef0-2051914c1c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.756951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520808ce-da4f-41e9-bef0-2051914c1c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.759314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520808ce-da4f-41e9-bef0-2051914c1c5e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.770682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1076/2164] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:23:51.774853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cbbb6-5b1a-4084-aa41-34680b95ce3b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.785997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cbbb6-5b1a-4084-aa41-34680b95ce3b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:51.795003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f476f7-63ae-4e8a-958a-f3353dc7908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:51.795682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f476f7-63ae-4e8a-958a-f3353dc7908e admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.795962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f476f7-63ae-4e8a-958a-f3353dc7908e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.796238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f476f7-63ae-4e8a-958a-f3353dc7908e admin admin] PATH_INFO: `/v3/projects/6121c8241c37402c88ec43864ea3c7b4/users/399186c0e20c4b51a6be3ef433eec005/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.798238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f476f7-63ae-4e8a-958a-f3353dc7908e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6121c8241c37402c88ec43864ea3c7b4, user_id=399186c0e20c4b51a6be3ef433eec005, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:51.800886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f476f7-63ae-4e8a-958a-f3353dc7908e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:51.816466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1089/2165] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:23:51 2022] PUT /identity/v3/projects/6121c8241c37402c88ec43864ea3c7b4/users/399186c0e20c4b51a6be3ef433eec005/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:23:51.827190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4790-61c6-4f8c-b403-4546e5d6fc25 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:51.827495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4790-61c6-4f8c-b403-4546e5d6fc25 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:51.827851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf4790-61c6-4f8c-b403-4546e5d6fc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:51.860023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8fcf4790-61c6-4f8c-b403-4546e5d6fc25 None None] MFA Rules not processed for user `399186c0e20c4b51a6be3ef433eec005`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:23:51.861102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8fcf4790-61c6-4f8c-b403-4546e5d6fc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:51.868770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:23:51.951673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1077/2166] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:23:51 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:23:51.962935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f476f7-63ae-4e8a-958a-f3353dc7908e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:51.964621 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e5f476f7-63ae-4e8a-958a-f3353dc7908e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:23:52.050777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f476f7-63ae-4e8a-958a-f3353dc7908e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:52.055488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9489e36-0e87-4f23-ab37-8876717fdf10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:52.056162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9489e36-0e87-4f23-ab37-8876717fdf10 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:52.056551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9489e36-0e87-4f23-ab37-8876717fdf10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:52.056876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9489e36-0e87-4f23-ab37-8876717fdf10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:52.073425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9489e36-0e87-4f23-ab37-8876717fdf10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:52.074958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9489e36-0e87-4f23-ab37-8876717fdf10 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:52.085576 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1090/2167] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:51 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:23:52.263673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf4790-61c6-4f8c-b403-4546e5d6fc25 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:23:52.273852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf4790-61c6-4f8c-b403-4546e5d6fc25 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:23:52.278304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f7c2d-83cc-4c3a-903f-fc45c2cbe208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:23:52.278990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f7c2d-83cc-4c3a-903f-fc45c2cbe208 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:23:52.279266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f7c2d-83cc-4c3a-903f-fc45c2cbe208 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:23:52.279542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f7c2d-83cc-4c3a-903f-fc45c2cbe208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:23:52.347088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f7c2d-83cc-4c3a-903f-fc45c2cbe208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:23:52.348653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f7c2d-83cc-4c3a-903f-fc45c2cbe208 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:23:52.358746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1078/2168] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:23:52 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:00.901901 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9489e36-0e87-4f23-ab37-8876717fdf10 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:00.930283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:00.966817 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9489e36-0e87-4f23-ab37-8876717fdf10 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:00.971372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:00.971997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:00.972278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:00.972551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] PATH_INFO: `/v3/users/3f0f9dd9d06d464d93df04b57e94abcf` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:00.973695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0f9dd9d06d464d93df04b57e94abcf)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:00.975630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:01.021430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0f9dd9d06d464d93df04b57e94abcf'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.022110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0f9dd9d06d464d93df04b57e94abcf'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.029638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0f9dd9d06d464d93df04b57e94abcf'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.030255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0f9dd9d06d464d93df04b57e94abcf'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.037420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0f9dd9d06d464d93df04b57e94abcf'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.054793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1091/2169] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:00 2022] DELETE /identity/v3/users/3f0f9dd9d06d464d93df04b57e94abcf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:01.059410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27503aad-97a2-418d-ad08-9743459de717 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:01.059914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27503aad-97a2-418d-ad08-9743459de717 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:01.060016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27503aad-97a2-418d-ad08-9743459de717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:01.079230 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-27503aad-97a2-418d-ad08-9743459de717 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:01.080447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-27503aad-97a2-418d-ad08-9743459de717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:01.139770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1079/2170] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:01.149619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:01.151038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:01.211152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8ef26-811d-4376-b36e-545801f3536c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:01.214769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eee764-d7a0-4e5b-949c-a3c8c44c95df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:01.215872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eee764-d7a0-4e5b-949c-a3c8c44c95df service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:01.216149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eee764-d7a0-4e5b-949c-a3c8c44c95df service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:01.216417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eee764-d7a0-4e5b-949c-a3c8c44c95df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:01.227148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eee764-d7a0-4e5b-949c-a3c8c44c95df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:01.228623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eee764-d7a0-4e5b-949c-a3c8c44c95df service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:01.238977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1092/2171] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:01.649273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27503aad-97a2-418d-ad08-9743459de717 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:01.650735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-27503aad-97a2-418d-ad08-9743459de717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:01.664633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27503aad-97a2-418d-ad08-9743459de717 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:01.670004 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:01.670754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:01.671130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:01.671410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] PATH_INFO: `/v3/projects/5a1b4f43e75d4169a7ef558fc2debce9` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:01.672866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1b4f43e75d4169a7ef558fc2debce9)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:01.680022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:01.721902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:01.734104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1b4f43e75d4169a7ef558fc2debce9', 'request_id': 'req-f6ae462c-9604-4297-ac1e-31868d09932f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.742697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1b4f43e75d4169a7ef558fc2debce9', 'request_id': 'req-f6ae462c-9604-4297-ac1e-31868d09932f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.749818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1b4f43e75d4169a7ef558fc2debce9', 'request_id': 'req-f6ae462c-9604-4297-ac1e-31868d09932f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.767399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] The token cache is being invalidate because project 5a1b4f43e75d4169a7ef558fc2debce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:01.768243 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.768558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.770167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1080/2172] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:01 2022] DELETE /identity/v3/projects/5a1b4f43e75d4169a7ef558fc2debce9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:01.774220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eee764-d7a0-4e5b-949c-a3c8c44c95df service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:01.784306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eee764-d7a0-4e5b-949c-a3c8c44c95df service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:01.826807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:01.844106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:01.845335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:01.845654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:01.845928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] PATH_INFO: `/v3/users/399186c0e20c4b51a6be3ef433eec005` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:01.847059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] RBAC: Authorizing `identity:delete_user(user_id=399186c0e20c4b51a6be3ef433eec005)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:01.848967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:01.908486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '399186c0e20c4b51a6be3ef433eec005'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.909502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '399186c0e20c4b51a6be3ef433eec005'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.917644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '399186c0e20c4b51a6be3ef433eec005'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.918969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '399186c0e20c4b51a6be3ef433eec005'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.930447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-25009536-59e2-4baf-ab6b-1013802bb65e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '399186c0e20c4b51a6be3ef433eec005'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:01.949527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1093/2173] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:01 2022] DELETE /identity/v3/users/399186c0e20c4b51a6be3ef433eec005 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:02.237826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:02.239719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:02.255167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae462c-9604-4297-ac1e-31868d09932f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:02.320863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:02.321644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.321999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.322270 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] PATH_INFO: `/v3/projects/6121c8241c37402c88ec43864ea3c7b4` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.323385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6121c8241c37402c88ec43864ea3c7b4)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:02.325931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:02.376483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6121c8241c37402c88ec43864ea3c7b4', 'request_id': 'req-65442190-e705-4202-b5eb-463c4310e5b4'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:02.384281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6121c8241c37402c88ec43864ea3c7b4', 'request_id': 'req-65442190-e705-4202-b5eb-463c4310e5b4'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:02.391348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6121c8241c37402c88ec43864ea3c7b4', 'request_id': 'req-65442190-e705-4202-b5eb-463c4310e5b4'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:02.402360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] The token cache is being invalidate because project 6121c8241c37402c88ec43864ea3c7b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:02.403276 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:02.403926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-65442190-e705-4202-b5eb-463c4310e5b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:02.405418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1081/2174] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:02 2022] DELETE /identity/v3/projects/6121c8241c37402c88ec43864ea3c7b4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:02.410228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b718a57-b2a3-4096-a801-d9ee7f6fa41a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.410461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b718a57-b2a3-4096-a801-d9ee7f6fa41a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.410740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b718a57-b2a3-4096-a801-d9ee7f6fa41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.430271 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5b718a57-b2a3-4096-a801-d9ee7f6fa41a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:02.431475 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5b718a57-b2a3-4096-a801-d9ee7f6fa41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:02.499889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1094/2175] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:02.501798 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35b9d-57b3-4356-a710-adbf8428fb36 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.502122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35b9d-57b3-4356-a710-adbf8428fb36 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.502461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35b9d-57b3-4356-a710-adbf8428fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.522749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-1ec35b9d-57b3-4356-a710-adbf8428fb36 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:02.523924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1ec35b9d-57b3-4356-a710-adbf8428fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:02.531705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1082/2176] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:24:02.536073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b718a57-b2a3-4096-a801-d9ee7f6fa41a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:02.545800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b718a57-b2a3-4096-a801-d9ee7f6fa41a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:02.550409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3be891-9a75-4bfc-a7d9-b86f8fdcc629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:02.551004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be891-9a75-4bfc-a7d9-b86f8fdcc629 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.551281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be891-9a75-4bfc-a7d9-b86f8fdcc629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.551550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be891-9a75-4bfc-a7d9-b86f8fdcc629 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.552193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3be891-9a75-4bfc-a7d9-b86f8fdcc629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:24:02.552493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3be891-9a75-4bfc-a7d9-b86f8fdcc629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:02.554122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3be891-9a75-4bfc-a7d9-b86f8fdcc629 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:02.567930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1095/2177] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:24:02.572518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec35b9d-57b3-4356-a710-adbf8428fb36 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:02.582721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec35b9d-57b3-4356-a710-adbf8428fb36 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:02.587509 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eedd0-88c0-4705-96fb-fd6664e93e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:02.587905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eedd0-88c0-4705-96fb-fd6664e93e6c admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.588176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eedd0-88c0-4705-96fb-fd6664e93e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.588442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44eedd0-88c0-4705-96fb-fd6664e93e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.589058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44eedd0-88c0-4705-96fb-fd6664e93e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:02.591030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44eedd0-88c0-4705-96fb-fd6664e93e6c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:02.618016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1083/2178] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:24:02 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:02.622664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3be891-9a75-4bfc-a7d9-b86f8fdcc629 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:02.634928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3be891-9a75-4bfc-a7d9-b86f8fdcc629 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:02.693004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:02.704163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67de7b-5d11-484c-9b8a-f8fd4e6e2a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:02.704822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67de7b-5d11-484c-9b8a-f8fd4e6e2a81 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.705101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67de7b-5d11-484c-9b8a-f8fd4e6e2a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.705414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67de7b-5d11-484c-9b8a-f8fd4e6e2a81 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.706434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67de7b-5d11-484c-9b8a-f8fd4e6e2a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:02.709404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67de7b-5d11-484c-9b8a-f8fd4e6e2a81 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:02.734151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1096/2179] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:24:02 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:02.739799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eedd0-88c0-4705-96fb-fd6664e93e6c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:02.754026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44eedd0-88c0-4705-96fb-fd6664e93e6c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:02.760375 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede5ca5-9d02-4f1e-9382-b43f77020daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:02.761198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede5ca5-9d02-4f1e-9382-b43f77020daa admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.761573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede5ca5-9d02-4f1e-9382-b43f77020daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.761938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede5ca5-9d02-4f1e-9382-b43f77020daa admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.763280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede5ca5-9d02-4f1e-9382-b43f77020daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:02.766089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede5ca5-9d02-4f1e-9382-b43f77020daa admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:02.778996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1084/2180] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:02.782376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67de7b-5d11-484c-9b8a-f8fd4e6e2a81 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:02.794064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67de7b-5d11-484c-9b8a-f8fd4e6e2a81 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:02.798972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdd186-a13b-41f0-8e8f-eea3b9edeaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:02.799571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd186-a13b-41f0-8e8f-eea3b9edeaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.799849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd186-a13b-41f0-8e8f-eea3b9edeaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.800119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd186-a13b-41f0-8e8f-eea3b9edeaa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.800762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdd186-a13b-41f0-8e8f-eea3b9edeaa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:02.802400 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdd186-a13b-41f0-8e8f-eea3b9edeaa2 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:02.813278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1097/2181] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:02.817639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede5ca5-9d02-4f1e-9382-b43f77020daa admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:02.831815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede5ca5-9d02-4f1e-9382-b43f77020daa admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:02.838568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6effe-24b7-43e6-a670-945b1e2ef7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:02.839234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6effe-24b7-43e6-a670-945b1e2ef7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.839516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6effe-24b7-43e6-a670-945b1e2ef7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.839785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6effe-24b7-43e6-a670-945b1e2ef7c7 admin admin] PATH_INFO: `/v3/projects/3d145389e4eb4b79891a9fa69316b42f/users/68dcb237837a42e88189ee83786e8493/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.852692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:02.856401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6effe-24b7-43e6-a670-945b1e2ef7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d145389e4eb4b79891a9fa69316b42f, user_id=68dcb237837a42e88189ee83786e8493, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:02.859897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6effe-24b7-43e6-a670-945b1e2ef7c7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:02.874553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1085/2182] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:02 2022] PUT /identity/v3/projects/3d145389e4eb4b79891a9fa69316b42f/users/68dcb237837a42e88189ee83786e8493/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:02.885731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e06f62-13ea-4e36-991a-4f854e172dec None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:02.886017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e06f62-13ea-4e36-991a-4f854e172dec None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:02.886438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e06f62-13ea-4e36-991a-4f854e172dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:02.915878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b1e06f62-13ea-4e36-991a-4f854e172dec None None] MFA Rules not processed for user `68dcb237837a42e88189ee83786e8493`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:02.917138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b1e06f62-13ea-4e36-991a-4f854e172dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:02.983593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1098/2183] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:02.988100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6effe-24b7-43e6-a670-945b1e2ef7c7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:03.049989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6effe-24b7-43e6-a670-945b1e2ef7c7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:03.054621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d939bf-1fd2-4756-b2c1-b8a30b4ef1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:03.055303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d939bf-1fd2-4756-b2c1-b8a30b4ef1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:03.055642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d939bf-1fd2-4756-b2c1-b8a30b4ef1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:03.055982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d939bf-1fd2-4756-b2c1-b8a30b4ef1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:03.056673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d939bf-1fd2-4756-b2c1-b8a30b4ef1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:03.058664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d939bf-1fd2-4756-b2c1-b8a30b4ef1d3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:03.088879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1086/2184] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:24:02 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:03.090827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e06f62-13ea-4e36-991a-4f854e172dec None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:03.101411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e06f62-13ea-4e36-991a-4f854e172dec None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:03.158037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d8d74-ecf2-48b0-ad10-eb2a6c784a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:03.158656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d8d74-ecf2-48b0-ad10-eb2a6c784a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:03.158927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d8d74-ecf2-48b0-ad10-eb2a6c784a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:03.159197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d8d74-ecf2-48b0-ad10-eb2a6c784a01 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:03.159822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d8d74-ecf2-48b0-ad10-eb2a6c784a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:03.161778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d8d74-ecf2-48b0-ad10-eb2a6c784a01 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:03.186920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1099/2185] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:24:03 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:03.187910 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d939bf-1fd2-4756-b2c1-b8a30b4ef1d3 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:03.199264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d939bf-1fd2-4756-b2c1-b8a30b4ef1d3 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:03.204299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ac630-1e49-4e9b-916c-bc2225cca42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:03.205092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ac630-1e49-4e9b-916c-bc2225cca42e admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:03.205476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ac630-1e49-4e9b-916c-bc2225cca42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:03.205802 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ac630-1e49-4e9b-916c-bc2225cca42e admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:03.206453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ac630-1e49-4e9b-916c-bc2225cca42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:03.208147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ac630-1e49-4e9b-916c-bc2225cca42e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:03.221553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1087/2186] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:03.225251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d8d74-ecf2-48b0-ad10-eb2a6c784a01 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:03.235844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d8d74-ecf2-48b0-ad10-eb2a6c784a01 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:03.240642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbca8cd-dc8a-43dd-8e3b-be737cd42cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:03.241080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbca8cd-dc8a-43dd-8e3b-be737cd42cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:03.241362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbca8cd-dc8a-43dd-8e3b-be737cd42cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:03.241679 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbca8cd-dc8a-43dd-8e3b-be737cd42cb9 admin admin] PATH_INFO: `/v3/projects/379e6c79d10d407f89956b61f8e695f3/users/9dbb63fa49b44c7d92ef7f716ede331c/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:03.258559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbca8cd-dc8a-43dd-8e3b-be737cd42cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=379e6c79d10d407f89956b61f8e695f3, user_id=9dbb63fa49b44c7d92ef7f716ede331c, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:03.261226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbca8cd-dc8a-43dd-8e3b-be737cd42cb9 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:03.275170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1100/2187] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:03 2022] PUT /identity/v3/projects/379e6c79d10d407f89956b61f8e695f3/users/9dbb63fa49b44c7d92ef7f716ede331c/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:24:03.279629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ac630-1e49-4e9b-916c-bc2225cca42e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:03.290094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ac630-1e49-4e9b-916c-bc2225cca42e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:03.348197 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38235ac-021f-4594-b4f0-967fa07c7d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:03.348812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38235ac-021f-4594-b4f0-967fa07c7d6c admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:03.349086 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38235ac-021f-4594-b4f0-967fa07c7d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:03.349354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38235ac-021f-4594-b4f0-967fa07c7d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:03.350019 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38235ac-021f-4594-b4f0-967fa07c7d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:03.351607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38235ac-021f-4594-b4f0-967fa07c7d6c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:03.361357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1088/2188] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:03.365447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbca8cd-dc8a-43dd-8e3b-be737cd42cb9 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:03.376082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbca8cd-dc8a-43dd-8e3b-be737cd42cb9 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:03.380644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783c67d-1760-4c5c-a2f2-c3df6ad21649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:03.381290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783c67d-1760-4c5c-a2f2-c3df6ad21649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:03.381596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783c67d-1760-4c5c-a2f2-c3df6ad21649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:03.381872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783c67d-1760-4c5c-a2f2-c3df6ad21649 admin admin] PATH_INFO: `/v3/projects/379e6c79d10d407f89956b61f8e695f3/users/9dbb63fa49b44c7d92ef7f716ede331c/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:03.383786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783c67d-1760-4c5c-a2f2-c3df6ad21649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=379e6c79d10d407f89956b61f8e695f3, user_id=9dbb63fa49b44c7d92ef7f716ede331c, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:03.386425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783c67d-1760-4c5c-a2f2-c3df6ad21649 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:03.398826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1101/2189] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:03 2022] PUT /identity/v3/projects/379e6c79d10d407f89956b61f8e695f3/users/9dbb63fa49b44c7d92ef7f716ede331c/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:03.409248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de36f7b-fbce-46ff-86b6-0ed6fdc26d37 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:03.409564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de36f7b-fbce-46ff-86b6-0ed6fdc26d37 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:03.409843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de36f7b-fbce-46ff-86b6-0ed6fdc26d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:03.440352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-1de36f7b-fbce-46ff-86b6-0ed6fdc26d37 None None] MFA Rules not processed for user `9dbb63fa49b44c7d92ef7f716ede331c`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:03.441490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1de36f7b-fbce-46ff-86b6-0ed6fdc26d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:03.491928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:03.515472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1089/2190] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:03.527341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783c67d-1760-4c5c-a2f2-c3df6ad21649 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:03.529877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4783c67d-1760-4c5c-a2f2-c3df6ad21649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:03.548547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:03.595288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783c67d-1760-4c5c-a2f2-c3df6ad21649 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:03.599043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fd01f-e2b0-4781-99ef-a7e7bbc0c696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:03.599657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fd01f-e2b0-4781-99ef-a7e7bbc0c696 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:03.599933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fd01f-e2b0-4781-99ef-a7e7bbc0c696 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:03.600628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fd01f-e2b0-4781-99ef-a7e7bbc0c696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:03.610827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fd01f-e2b0-4781-99ef-a7e7bbc0c696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:03.612554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fd01f-e2b0-4781-99ef-a7e7bbc0c696 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:03.625022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1102/2191] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:04.104577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de36f7b-fbce-46ff-86b6-0ed6fdc26d37 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:04.186344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de36f7b-fbce-46ff-86b6-0ed6fdc26d37 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:04.190919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:04.191562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:04.191840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:04.192379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] PATH_INFO: `/v3/users/68dcb237837a42e88189ee83786e8493` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:04.193557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68dcb237837a42e88189ee83786e8493)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:04.195481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:04.245272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68dcb237837a42e88189ee83786e8493'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.245953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68dcb237837a42e88189ee83786e8493'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.253759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68dcb237837a42e88189ee83786e8493'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.261553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68dcb237837a42e88189ee83786e8493'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.263009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68dcb237837a42e88189ee83786e8493'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.282053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1090/2192] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:04 2022] DELETE /identity/v3/users/68dcb237837a42e88189ee83786e8493 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:04.287404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77db3c-9fce-4b43-b940-b49a43d86aa1 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:04.288173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77db3c-9fce-4b43-b940-b49a43d86aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:04.288173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77db3c-9fce-4b43-b940-b49a43d86aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:04.308114 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-fa77db3c-9fce-4b43-b940-b49a43d86aa1 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:04.309393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fa77db3c-9fce-4b43-b940-b49a43d86aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:04.371929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1103/2193] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:04.382042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:04.383510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:04.443768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a1336-4426-47bf-8ec3-f443cd5b85d0 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:04.447477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d17fe8-8845-4812-86a0-74ac83156000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:04.448088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17fe8-8845-4812-86a0-74ac83156000 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:04.448368 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17fe8-8845-4812-86a0-74ac83156000 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:04.448643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d17fe8-8845-4812-86a0-74ac83156000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:04.459141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d17fe8-8845-4812-86a0-74ac83156000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:04.460581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d17fe8-8845-4812-86a0-74ac83156000 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:04.471047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1091/2194] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:04.749451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77db3c-9fce-4b43-b940-b49a43d86aa1 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:04.750931 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fa77db3c-9fce-4b43-b940-b49a43d86aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:04.764175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77db3c-9fce-4b43-b940-b49a43d86aa1 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:04.770766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:04.771581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:04.771955 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:04.772323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] PATH_INFO: `/v3/projects/3d145389e4eb4b79891a9fa69316b42f` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:04.773641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d145389e4eb4b79891a9fa69316b42f)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:04.776533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:04.828733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d145389e4eb4b79891a9fa69316b42f', 'request_id': 'req-285351e5-5d53-48f3-a82d-33964b2e0a10'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.837862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d145389e4eb4b79891a9fa69316b42f', 'request_id': 'req-285351e5-5d53-48f3-a82d-33964b2e0a10'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.846316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d145389e4eb4b79891a9fa69316b42f', 'request_id': 'req-285351e5-5d53-48f3-a82d-33964b2e0a10'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.858947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] The token cache is being invalidate because project 3d145389e4eb4b79891a9fa69316b42f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:04.859199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.860150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:04.866066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1104/2195] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:04 2022] DELETE /identity/v3/projects/3d145389e4eb4b79891a9fa69316b42f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:04.866254 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d17fe8-8845-4812-86a0-74ac83156000 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:04.877850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d17fe8-8845-4812-86a0-74ac83156000 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:04.887310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:04.946894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:04.947582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:04.947858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:04.948129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] PATH_INFO: `/v3/users/9dbb63fa49b44c7d92ef7f716ede331c` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:04.949242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dbb63fa49b44c7d92ef7f716ede331c)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:04.951353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:05.006517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dbb63fa49b44c7d92ef7f716ede331c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.007349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dbb63fa49b44c7d92ef7f716ede331c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.014788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dbb63fa49b44c7d92ef7f716ede331c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.022865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dbb63fa49b44c7d92ef7f716ede331c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.023773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-0e1f4c07-7a38-49c3-b269-50b1c5951fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dbb63fa49b44c7d92ef7f716ede331c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.044935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1092/2196] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:04 2022] DELETE /identity/v3/users/9dbb63fa49b44c7d92ef7f716ede331c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:05.418504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:05.420070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:05.434593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285351e5-5d53-48f3-a82d-33964b2e0a10 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:05.476286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:05.497135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:05.498052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:05.498329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:05.498615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] PATH_INFO: `/v3/projects/379e6c79d10d407f89956b61f8e695f3` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:05.499766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=379e6c79d10d407f89956b61f8e695f3)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:05.501693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:05.546748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '379e6c79d10d407f89956b61f8e695f3', 'request_id': 'req-183a6bdd-c3f0-4eef-a113-f26865701ca1'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.553987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '379e6c79d10d407f89956b61f8e695f3', 'request_id': 'req-183a6bdd-c3f0-4eef-a113-f26865701ca1'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.561705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '379e6c79d10d407f89956b61f8e695f3', 'request_id': 'req-183a6bdd-c3f0-4eef-a113-f26865701ca1'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.574126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] The token cache is being invalidate because project 379e6c79d10d407f89956b61f8e695f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:05.574974 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.575820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-183a6bdd-c3f0-4eef-a113-f26865701ca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:05.581046 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1105/2197] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:05 2022] DELETE /identity/v3/projects/379e6c79d10d407f89956b61f8e695f3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:24:05.582644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a222a2-6c3f-43f0-8934-126b616787da None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:05.583100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a222a2-6c3f-43f0-8934-126b616787da None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:05.583180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a222a2-6c3f-43f0-8934-126b616787da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:05.604295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-61a222a2-6c3f-43f0-8934-126b616787da None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:05.605459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-61a222a2-6c3f-43f0-8934-126b616787da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:05.689025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1093/2198] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:05.697912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc2430-5e58-4f1b-838c-71aca92275bc None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:05.698072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc2430-5e58-4f1b-838c-71aca92275bc None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:05.698216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc2430-5e58-4f1b-838c-71aca92275bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:05.721910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-09cc2430-5e58-4f1b-838c-71aca92275bc None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:05.723219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-09cc2430-5e58-4f1b-838c-71aca92275bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:05.732490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1106/2199] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:05.736593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a222a2-6c3f-43f0-8934-126b616787da None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:05.747779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a222a2-6c3f-43f0-8934-126b616787da None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:05.753339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd9ecb-3905-4cce-9b74-2fe217e84b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:05.753999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd9ecb-3905-4cce-9b74-2fe217e84b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:05.754416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd9ecb-3905-4cce-9b74-2fe217e84b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:05.754679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd9ecb-3905-4cce-9b74-2fe217e84b8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:05.755340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd9ecb-3905-4cce-9b74-2fe217e84b8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:24:05.755635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd9ecb-3905-4cce-9b74-2fe217e84b8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:05.757305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd9ecb-3905-4cce-9b74-2fe217e84b8a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:05.771359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1094/2200] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:24:05.775311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc2430-5e58-4f1b-838c-71aca92275bc None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:05.785020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc2430-5e58-4f1b-838c-71aca92275bc None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:05.789696 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf1c59-72c5-4f26-ae6c-27563588ef42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:05.790288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1c59-72c5-4f26-ae6c-27563588ef42 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:05.790561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1c59-72c5-4f26-ae6c-27563588ef42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:05.790828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1c59-72c5-4f26-ae6c-27563588ef42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:05.791447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf1c59-72c5-4f26-ae6c-27563588ef42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:05.793927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf1c59-72c5-4f26-ae6c-27563588ef42 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:05.825923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1107/2201] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:24:05 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:05.828013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd9ecb-3905-4cce-9b74-2fe217e84b8a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:05.839023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd9ecb-3905-4cce-9b74-2fe217e84b8a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:05.851991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:05.912984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541284a-d579-4ebc-b639-fb69647457fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:05.913751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541284a-d579-4ebc-b639-fb69647457fc admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:05.914112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541284a-d579-4ebc-b639-fb69647457fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:05.914452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541284a-d579-4ebc-b639-fb69647457fc admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:05.915162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541284a-d579-4ebc-b639-fb69647457fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:05.917190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541284a-d579-4ebc-b639-fb69647457fc admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:05.943063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1095/2202] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:24:05 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:05.947427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf1c59-72c5-4f26-ae6c-27563588ef42 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:05.958413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf1c59-72c5-4f26-ae6c-27563588ef42 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:05.963855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ac084-5f95-4660-a25c-30dabfb6cd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:05.964488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ac084-5f95-4660-a25c-30dabfb6cd89 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:05.964765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ac084-5f95-4660-a25c-30dabfb6cd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:05.965674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ac084-5f95-4660-a25c-30dabfb6cd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:05.966328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ac084-5f95-4660-a25c-30dabfb6cd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:05.967918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280ac084-5f95-4660-a25c-30dabfb6cd89 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:05.978435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1108/2203] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:24:05.982417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541284a-d579-4ebc-b639-fb69647457fc admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:05.992431 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541284a-d579-4ebc-b639-fb69647457fc admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:05.997000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643bbab-15dc-4bc1-999d-d930010cdfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:05.997627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643bbab-15dc-4bc1-999d-d930010cdfed admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:05.997905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643bbab-15dc-4bc1-999d-d930010cdfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:05.998201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643bbab-15dc-4bc1-999d-d930010cdfed admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:05.998808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643bbab-15dc-4bc1-999d-d930010cdfed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:06.000385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643bbab-15dc-4bc1-999d-d930010cdfed admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:06.011613 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1096/2204] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:06.016282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ac084-5f95-4660-a25c-30dabfb6cd89 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:06.028212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ac084-5f95-4660-a25c-30dabfb6cd89 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:06.033267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb3107-ff94-4bef-82ab-6f4b6b026629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:06.034062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb3107-ff94-4bef-82ab-6f4b6b026629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.034440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb3107-ff94-4bef-82ab-6f4b6b026629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.034811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb3107-ff94-4bef-82ab-6f4b6b026629 admin admin] PATH_INFO: `/v3/projects/fd3db8415d3c4691994d95c5c26a29b2/users/2d542046ea674484aacf4e2b78ec85db/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.052226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb3107-ff94-4bef-82ab-6f4b6b026629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3db8415d3c4691994d95c5c26a29b2, user_id=2d542046ea674484aacf4e2b78ec85db, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:06.055945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb3107-ff94-4bef-82ab-6f4b6b026629 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:06.070052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1109/2205] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:06 2022] PUT /identity/v3/projects/fd3db8415d3c4691994d95c5c26a29b2/users/2d542046ea674484aacf4e2b78ec85db/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:06.080478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0457403-7764-42a9-aed3-495694d8c6ad None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.080765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0457403-7764-42a9-aed3-495694d8c6ad None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.081061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0457403-7764-42a9-aed3-495694d8c6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.109897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-d0457403-7764-42a9-aed3-495694d8c6ad None None] MFA Rules not processed for user `2d542046ea674484aacf4e2b78ec85db`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:06.111034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d0457403-7764-42a9-aed3-495694d8c6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:06.179210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1097/2206] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:06.184166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb3107-ff94-4bef-82ab-6f4b6b026629 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:06.249646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb3107-ff94-4bef-82ab-6f4b6b026629 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:06.254445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699c8bf-3917-49f7-9d8e-54c70af1b53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:06.255053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c8bf-3917-49f7-9d8e-54c70af1b53b admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.255614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c8bf-3917-49f7-9d8e-54c70af1b53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.255743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c8bf-3917-49f7-9d8e-54c70af1b53b admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.256308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699c8bf-3917-49f7-9d8e-54c70af1b53b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:06.258556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699c8bf-3917-49f7-9d8e-54c70af1b53b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:06.270566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:06.289409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1110/2207] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:24:06 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:06.293941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0457403-7764-42a9-aed3-495694d8c6ad None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:06.304742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0457403-7764-42a9-aed3-495694d8c6ad None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:06.359639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39879af5-94d9-48df-9300-5576f4d712d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:06.360294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39879af5-94d9-48df-9300-5576f4d712d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.360520 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39879af5-94d9-48df-9300-5576f4d712d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.360796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39879af5-94d9-48df-9300-5576f4d712d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.361421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39879af5-94d9-48df-9300-5576f4d712d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:06.363350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39879af5-94d9-48df-9300-5576f4d712d2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:06.388001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1098/2208] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:24:06 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:24:06.389033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699c8bf-3917-49f7-9d8e-54c70af1b53b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:06.399889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699c8bf-3917-49f7-9d8e-54c70af1b53b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:06.404681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741db96-e927-4a7e-a934-9a7df4dbcc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:06.405283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741db96-e927-4a7e-a934-9a7df4dbcc8d admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.405662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741db96-e927-4a7e-a934-9a7df4dbcc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.405941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741db96-e927-4a7e-a934-9a7df4dbcc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.406580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741db96-e927-4a7e-a934-9a7df4dbcc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:06.408198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741db96-e927-4a7e-a934-9a7df4dbcc8d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:06.419487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1111/2209] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:24:06.423195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39879af5-94d9-48df-9300-5576f4d712d2 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:06.433198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39879af5-94d9-48df-9300-5576f4d712d2 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:06.441268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b50eb-1374-429f-821a-8ce3e58bb5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:06.442246 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b50eb-1374-429f-821a-8ce3e58bb5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.442523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b50eb-1374-429f-821a-8ce3e58bb5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.442797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b50eb-1374-429f-821a-8ce3e58bb5fb admin admin] PATH_INFO: `/v3/projects/19f82776e2ff44b287dc17c8aaf512d0/users/31575f1f5702479d84f99a01a03038c5/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.455198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:06.463623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b50eb-1374-429f-821a-8ce3e58bb5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f82776e2ff44b287dc17c8aaf512d0, user_id=31575f1f5702479d84f99a01a03038c5, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:06.466635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b50eb-1374-429f-821a-8ce3e58bb5fb admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:06.482106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1099/2210] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:06 2022] PUT /identity/v3/projects/19f82776e2ff44b287dc17c8aaf512d0/users/31575f1f5702479d84f99a01a03038c5/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:06.486095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741db96-e927-4a7e-a934-9a7df4dbcc8d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:06.498297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741db96-e927-4a7e-a934-9a7df4dbcc8d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:06.571800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899fa79-ff06-4d21-a19f-784d3a0b2601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:06.572415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899fa79-ff06-4d21-a19f-784d3a0b2601 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.572672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899fa79-ff06-4d21-a19f-784d3a0b2601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.572937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899fa79-ff06-4d21-a19f-784d3a0b2601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.573611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899fa79-ff06-4d21-a19f-784d3a0b2601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:06.586830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899fa79-ff06-4d21-a19f-784d3a0b2601 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:06.597585 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1112/2211] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:06.600796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b50eb-1374-429f-821a-8ce3e58bb5fb admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:06.611762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b50eb-1374-429f-821a-8ce3e58bb5fb admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:06.616263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77298dd9-fb4a-40cd-8603-4de49143fed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:06.616913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77298dd9-fb4a-40cd-8603-4de49143fed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.617207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77298dd9-fb4a-40cd-8603-4de49143fed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.617461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77298dd9-fb4a-40cd-8603-4de49143fed8 admin admin] PATH_INFO: `/v3/projects/19f82776e2ff44b287dc17c8aaf512d0/users/31575f1f5702479d84f99a01a03038c5/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.619377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77298dd9-fb4a-40cd-8603-4de49143fed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f82776e2ff44b287dc17c8aaf512d0, user_id=31575f1f5702479d84f99a01a03038c5, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:06.622651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77298dd9-fb4a-40cd-8603-4de49143fed8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:06.634405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1100/2212] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:06 2022] PUT /identity/v3/projects/19f82776e2ff44b287dc17c8aaf512d0/users/31575f1f5702479d84f99a01a03038c5/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:06.644809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217fd15-147b-4b57-bc1b-8ebfeda5e0ae None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.645094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217fd15-147b-4b57-bc1b-8ebfeda5e0ae None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.645407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217fd15-147b-4b57-bc1b-8ebfeda5e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.672868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c217fd15-147b-4b57-bc1b-8ebfeda5e0ae None None] MFA Rules not processed for user `31575f1f5702479d84f99a01a03038c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:06.674321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c217fd15-147b-4b57-bc1b-8ebfeda5e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:06.741262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1113/2213] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:06.756806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77298dd9-fb4a-40cd-8603-4de49143fed8 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:06.759334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-77298dd9-fb4a-40cd-8603-4de49143fed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:06.815808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77298dd9-fb4a-40cd-8603-4de49143fed8 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:06.819703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f983854-fa59-4c76-bde2-327c72e46dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:06.820303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f983854-fa59-4c76-bde2-327c72e46dee service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.820578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f983854-fa59-4c76-bde2-327c72e46dee service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.820846 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f983854-fa59-4c76-bde2-327c72e46dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:06.830875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f983854-fa59-4c76-bde2-327c72e46dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:06.832268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f983854-fa59-4c76-bde2-327c72e46dee service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:06.842355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1101/2214] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:06.936900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217fd15-147b-4b57-bc1b-8ebfeda5e0ae None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:06.946962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217fd15-147b-4b57-bc1b-8ebfeda5e0ae None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:06.951443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccac4d2-8bcc-466d-ba5d-03e939c46801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:06.952053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccac4d2-8bcc-466d-ba5d-03e939c46801 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:06.952332 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccac4d2-8bcc-466d-ba5d-03e939c46801 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:06.952608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccac4d2-8bcc-466d-ba5d-03e939c46801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:07.010075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccac4d2-8bcc-466d-ba5d-03e939c46801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:07.011524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccac4d2-8bcc-466d-ba5d-03e939c46801 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:07.016761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:07.022022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1114/2215] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:07.308159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f983854-fa59-4c76-bde2-327c72e46dee service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:07.373339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f983854-fa59-4c76-bde2-327c72e46dee service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:07.377772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:07.378388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:07.378661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:07.378927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] PATH_INFO: `/v3/users/2d542046ea674484aacf4e2b78ec85db` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:07.380018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d542046ea674484aacf4e2b78ec85db)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:07.381966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:07.426536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d542046ea674484aacf4e2b78ec85db'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:07.427102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d542046ea674484aacf4e2b78ec85db'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:07.434467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d542046ea674484aacf4e2b78ec85db'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:07.441671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d542046ea674484aacf4e2b78ec85db'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:07.442533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d542046ea674484aacf4e2b78ec85db'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:07.461402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:07.464904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1102/2216] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:07 2022] DELETE /identity/v3/users/2d542046ea674484aacf4e2b78ec85db => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:24:07.469689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36fc1-131c-4a37-b940-60e525ce3dd8 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:07.470264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36fc1-131c-4a37-b940-60e525ce3dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:07.470418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a36fc1-131c-4a37-b940-60e525ce3dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:07.490618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-93a36fc1-131c-4a37-b940-60e525ce3dd8 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:07.491906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-93a36fc1-131c-4a37-b940-60e525ce3dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:07.562302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1115/2217] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:24:07.572659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:07.574213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:07.637578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea567891-9a14-4318-9816-e44d6b21761c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:07.641660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060d4ff-e573-4e42-8931-4f378aca5c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:07.642268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060d4ff-e573-4e42-8931-4f378aca5c2d service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:07.642541 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060d4ff-e573-4e42-8931-4f378aca5c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:07.642868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060d4ff-e573-4e42-8931-4f378aca5c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:07.653305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060d4ff-e573-4e42-8931-4f378aca5c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:07.654826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060d4ff-e573-4e42-8931-4f378aca5c2d service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:07.664864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1103/2218] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:08.034081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a36fc1-131c-4a37-b940-60e525ce3dd8 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:08.035517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-93a36fc1-131c-4a37-b940-60e525ce3dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:08.047486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a36fc1-131c-4a37-b940-60e525ce3dd8 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:08.052213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:08.052717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:08.053002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:08.053279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] PATH_INFO: `/v3/projects/fd3db8415d3c4691994d95c5c26a29b2` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:08.054410 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3db8415d3c4691994d95c5c26a29b2)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:08.056328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:08.118233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3db8415d3c4691994d95c5c26a29b2', 'request_id': 'req-3493164d-d6a9-4f59-8ae5-503f93f5761d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.130163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3db8415d3c4691994d95c5c26a29b2', 'request_id': 'req-3493164d-d6a9-4f59-8ae5-503f93f5761d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.136205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3db8415d3c4691994d95c5c26a29b2', 'request_id': 'req-3493164d-d6a9-4f59-8ae5-503f93f5761d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.149931 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] The token cache is being invalidate because project fd3db8415d3c4691994d95c5c26a29b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:08.150268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.151129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.153272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1116/2219] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:08 2022] DELETE /identity/v3/projects/fd3db8415d3c4691994d95c5c26a29b2 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:08.157178 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060d4ff-e573-4e42-8931-4f378aca5c2d service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:08.167296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060d4ff-e573-4e42-8931-4f378aca5c2d service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:08.222864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:08.223399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:08.223673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:08.223937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] PATH_INFO: `/v3/users/31575f1f5702479d84f99a01a03038c5` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:08.225030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31575f1f5702479d84f99a01a03038c5)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:08.226943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:08.272149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31575f1f5702479d84f99a01a03038c5'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.272793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31575f1f5702479d84f99a01a03038c5'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.279859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31575f1f5702479d84f99a01a03038c5'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.287043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31575f1f5702479d84f99a01a03038c5'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.287655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9262f56-a780-4ce3-89bf-883b55a046a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31575f1f5702479d84f99a01a03038c5'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.305771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1104/2220] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:08 2022] DELETE /identity/v3/users/31575f1f5702479d84f99a01a03038c5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:08.572535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:08.574108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:08.588313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493164d-d6a9-4f59-8ae5-503f93f5761d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:08.645961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:08.646586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:08.646861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:08.647131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] PATH_INFO: `/v3/projects/19f82776e2ff44b287dc17c8aaf512d0` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:08.648230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f82776e2ff44b287dc17c8aaf512d0)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:08.650196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:08.679171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:08.697403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f82776e2ff44b287dc17c8aaf512d0', 'request_id': 'req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.705265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f82776e2ff44b287dc17c8aaf512d0', 'request_id': 'req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.711435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f82776e2ff44b287dc17c8aaf512d0', 'request_id': 'req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.721492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] The token cache is being invalidate because project 19f82776e2ff44b287dc17c8aaf512d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:08.721806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.722427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-45dc9734-8e20-4d6c-a79b-ab8f1e64952c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:08.723889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1117/2221] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:08 2022] DELETE /identity/v3/projects/19f82776e2ff44b287dc17c8aaf512d0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:08.728681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d64d9-7c43-46c1-bb4e-fef443aa0a3a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:08.729151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d64d9-7c43-46c1-bb4e-fef443aa0a3a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:08.729245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d64d9-7c43-46c1-bb4e-fef443aa0a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:08.749087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-3d0d64d9-7c43-46c1-bb4e-fef443aa0a3a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:08.750514 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3d0d64d9-7c43-46c1-bb4e-fef443aa0a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:08.763779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:08.811775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1105/2222] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:08.820051 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15219640-c963-4aef-90ab-b70788a9d34e None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:08.820334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15219640-c963-4aef-90ab-b70788a9d34e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:08.820608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15219640-c963-4aef-90ab-b70788a9d34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:08.842101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-15219640-c963-4aef-90ab-b70788a9d34e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:08.843335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-15219640-c963-4aef-90ab-b70788a9d34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:08.856845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1118/2223] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:08.858169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d64d9-7c43-46c1-bb4e-fef443aa0a3a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:08.869490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d64d9-7c43-46c1-bb4e-fef443aa0a3a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:08.874405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292010f-4c08-4710-8614-c12de0bd7e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:08.875011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292010f-4c08-4710-8614-c12de0bd7e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:08.875287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292010f-4c08-4710-8614-c12de0bd7e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:08.875555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292010f-4c08-4710-8614-c12de0bd7e7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:08.876353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292010f-4c08-4710-8614-c12de0bd7e7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:24:08.876655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292010f-4c08-4710-8614-c12de0bd7e7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:08.878385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292010f-4c08-4710-8614-c12de0bd7e7c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:08.895437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1106/2224] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:24:08.899512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15219640-c963-4aef-90ab-b70788a9d34e None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:08.909612 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15219640-c963-4aef-90ab-b70788a9d34e None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:08.914236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ff917-25b0-4049-9d6b-da10e6596688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:08.914836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ff917-25b0-4049-9d6b-da10e6596688 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:08.915108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ff917-25b0-4049-9d6b-da10e6596688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:08.915374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ff917-25b0-4049-9d6b-da10e6596688 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:08.916008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ff917-25b0-4049-9d6b-da10e6596688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:08.918005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ff917-25b0-4049-9d6b-da10e6596688 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:08.946409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1119/2225] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:24:08 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:24:08.951236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292010f-4c08-4710-8614-c12de0bd7e7c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:08.961077 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292010f-4c08-4710-8614-c12de0bd7e7c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.017469 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cbd68-b0d5-47fb-8fec-c3935f1e41cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.018027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cbd68-b0d5-47fb-8fec-c3935f1e41cf admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.018309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cbd68-b0d5-47fb-8fec-c3935f1e41cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.018581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cbd68-b0d5-47fb-8fec-c3935f1e41cf admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.019215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cbd68-b0d5-47fb-8fec-c3935f1e41cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.021119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cbd68-b0d5-47fb-8fec-c3935f1e41cf admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.046357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1107/2226] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:24:08 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:09.047652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ff917-25b0-4049-9d6b-da10e6596688 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.058116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ff917-25b0-4049-9d6b-da10e6596688 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.062702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ae802-f0df-48a5-afb9-7b31475bd0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.063290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae802-f0df-48a5-afb9-7b31475bd0da admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.063618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae802-f0df-48a5-afb9-7b31475bd0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.063874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae802-f0df-48a5-afb9-7b31475bd0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.064607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ae802-f0df-48a5-afb9-7b31475bd0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.066227 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ae802-f0df-48a5-afb9-7b31475bd0da admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.077083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1120/2227] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:09.081258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cbd68-b0d5-47fb-8fec-c3935f1e41cf admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.092481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cbd68-b0d5-47fb-8fec-c3935f1e41cf admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.097722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d403dda-3d8e-4b2a-b8e1-4383e02b370a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.098126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d403dda-3d8e-4b2a-b8e1-4383e02b370a admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.098403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d403dda-3d8e-4b2a-b8e1-4383e02b370a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.098674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d403dda-3d8e-4b2a-b8e1-4383e02b370a admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.099327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d403dda-3d8e-4b2a-b8e1-4383e02b370a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.100910 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d403dda-3d8e-4b2a-b8e1-4383e02b370a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.113141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1108/2228] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:09.117316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ae802-f0df-48a5-afb9-7b31475bd0da admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.129566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ae802-f0df-48a5-afb9-7b31475bd0da admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.133939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046105b-c21a-4480-872c-9ede40eab9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.134592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046105b-c21a-4480-872c-9ede40eab9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.134886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046105b-c21a-4480-872c-9ede40eab9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.135159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046105b-c21a-4480-872c-9ede40eab9e1 admin admin] PATH_INFO: `/v3/projects/12092a09a40c4d8b8bd6e857d981fa84/users/e2d0c91f08d1415fab87984d8a5ac339/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.151024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046105b-c21a-4480-872c-9ede40eab9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12092a09a40c4d8b8bd6e857d981fa84, user_id=e2d0c91f08d1415fab87984d8a5ac339, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.154241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046105b-c21a-4480-872c-9ede40eab9e1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.168938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1121/2229] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:09 2022] PUT /identity/v3/projects/12092a09a40c4d8b8bd6e857d981fa84/users/e2d0c91f08d1415fab87984d8a5ac339/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:09.179253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4e37-cd23-404f-b8c5-590ae33c2d6a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.179541 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4e37-cd23-404f-b8c5-590ae33c2d6a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.179813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4e37-cd23-404f-b8c5-590ae33c2d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.210651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-6d8b4e37-cd23-404f-b8c5-590ae33c2d6a None None] MFA Rules not processed for user `e2d0c91f08d1415fab87984d8a5ac339`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:09.211869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6d8b4e37-cd23-404f-b8c5-590ae33c2d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:09.284755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1109/2230] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:09.289118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046105b-c21a-4480-872c-9ede40eab9e1 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.351195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046105b-c21a-4480-872c-9ede40eab9e1 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.365807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39052096-06d1-4020-a102-a4879e4472c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.366653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39052096-06d1-4020-a102-a4879e4472c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.367016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39052096-06d1-4020-a102-a4879e4472c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.367330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39052096-06d1-4020-a102-a4879e4472c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.368141 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39052096-06d1-4020-a102-a4879e4472c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.370626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39052096-06d1-4020-a102-a4879e4472c0 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.397777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1122/2231] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:24:09 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:09.401914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b4e37-cd23-404f-b8c5-590ae33c2d6a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.413664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b4e37-cd23-404f-b8c5-590ae33c2d6a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.427008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:09.481754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd822f38-300a-4009-bdc6-460557477966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.482376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd822f38-300a-4009-bdc6-460557477966 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.482930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd822f38-300a-4009-bdc6-460557477966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.483052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd822f38-300a-4009-bdc6-460557477966 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.483608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd822f38-300a-4009-bdc6-460557477966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.485567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd822f38-300a-4009-bdc6-460557477966 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.509386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1110/2232] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:24:09 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:09.514429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39052096-06d1-4020-a102-a4879e4472c0 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.526982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39052096-06d1-4020-a102-a4879e4472c0 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.532145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18b70c-651d-4cd9-9220-edb75304260b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.532749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18b70c-651d-4cd9-9220-edb75304260b admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.533021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18b70c-651d-4cd9-9220-edb75304260b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.533291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18b70c-651d-4cd9-9220-edb75304260b admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.533969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18b70c-651d-4cd9-9220-edb75304260b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.535622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18b70c-651d-4cd9-9220-edb75304260b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.547989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1123/2233] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:09.552890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd822f38-300a-4009-bdc6-460557477966 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.565362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd822f38-300a-4009-bdc6-460557477966 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.571868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e69dd-41b6-41ed-9087-45a03584747d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.572554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e69dd-41b6-41ed-9087-45a03584747d admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.572833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e69dd-41b6-41ed-9087-45a03584747d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.573085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e69dd-41b6-41ed-9087-45a03584747d admin admin] PATH_INFO: `/v3/projects/553cb799de5f4cf3a170e778c343ba50/users/c50489b9a13044efa104762afedb0205/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.590625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e69dd-41b6-41ed-9087-45a03584747d admin admin] RBAC: Authorizing `identity:create_grant(project_id=553cb799de5f4cf3a170e778c343ba50, user_id=c50489b9a13044efa104762afedb0205, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.593570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e69dd-41b6-41ed-9087-45a03584747d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.618738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1111/2234] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:09 2022] PUT /identity/v3/projects/553cb799de5f4cf3a170e778c343ba50/users/c50489b9a13044efa104762afedb0205/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:09.619941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18b70c-651d-4cd9-9220-edb75304260b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.630926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18b70c-651d-4cd9-9220-edb75304260b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.655486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:09.711675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663eed5-fd23-457a-a1ec-681e74331978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.712286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663eed5-fd23-457a-a1ec-681e74331978 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.712559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663eed5-fd23-457a-a1ec-681e74331978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.712834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663eed5-fd23-457a-a1ec-681e74331978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.713460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663eed5-fd23-457a-a1ec-681e74331978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.715181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663eed5-fd23-457a-a1ec-681e74331978 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.725729 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1124/2235] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:09.729739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e69dd-41b6-41ed-9087-45a03584747d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.743060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e69dd-41b6-41ed-9087-45a03584747d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.749314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0e897-3640-4d51-9a22-f9932f8f5ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.750148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0e897-3640-4d51-9a22-f9932f8f5ada admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.750425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0e897-3640-4d51-9a22-f9932f8f5ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.750714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0e897-3640-4d51-9a22-f9932f8f5ada admin admin] PATH_INFO: `/v3/projects/553cb799de5f4cf3a170e778c343ba50/users/c50489b9a13044efa104762afedb0205/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.752782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0e897-3640-4d51-9a22-f9932f8f5ada admin admin] RBAC: Authorizing `identity:create_grant(project_id=553cb799de5f4cf3a170e778c343ba50, user_id=c50489b9a13044efa104762afedb0205, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:09.755329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0e897-3640-4d51-9a22-f9932f8f5ada admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:09.769069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1112/2236] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:09 2022] PUT /identity/v3/projects/553cb799de5f4cf3a170e778c343ba50/users/c50489b9a13044efa104762afedb0205/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:09.781272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cf5f8-65fa-4f21-9b6d-6aaa092e3269 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.781616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cf5f8-65fa-4f21-9b6d-6aaa092e3269 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.781897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cf5f8-65fa-4f21-9b6d-6aaa092e3269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:09.815587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b23cf5f8-65fa-4f21-9b6d-6aaa092e3269 None None] MFA Rules not processed for user `c50489b9a13044efa104762afedb0205`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:09.816870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b23cf5f8-65fa-4f21-9b6d-6aaa092e3269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:09.888251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1125/2237] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:09.899449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0e897-3640-4d51-9a22-f9932f8f5ada admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:09.901168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-22b0e897-3640-4d51-9a22-f9932f8f5ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:09.965058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0e897-3640-4d51-9a22-f9932f8f5ada admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:09.969174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000f765-aeb5-44b3-8631-2dfa41a4125a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:09.969804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000f765-aeb5-44b3-8631-2dfa41a4125a service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:09.970092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000f765-aeb5-44b3-8631-2dfa41a4125a service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:09.970349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000f765-aeb5-44b3-8631-2dfa41a4125a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:10.036366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000f765-aeb5-44b3-8631-2dfa41a4125a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:10.037877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000f765-aeb5-44b3-8631-2dfa41a4125a service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:10.047075 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1113/2238] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:14.331588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cf5f8-65fa-4f21-9b6d-6aaa092e3269 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:14.332990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b23cf5f8-65fa-4f21-9b6d-6aaa092e3269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:14.345279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cf5f8-65fa-4f21-9b6d-6aaa092e3269 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:14.348747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf9f5f-086e-4ca7-88d2-9073caf99026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:14.349340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf9f5f-086e-4ca7-88d2-9073caf99026 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:14.349660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf9f5f-086e-4ca7-88d2-9073caf99026 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:14.349934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf9f5f-086e-4ca7-88d2-9073caf99026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:14.359121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf9f5f-086e-4ca7-88d2-9073caf99026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:14.360530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf9f5f-086e-4ca7-88d2-9073caf99026 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:14.369906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1126/2239] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:20.042838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=85,keystone:INSERT=5 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:21.096936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e9db9-8fe4-416a-87d6-de38ef45f9d0 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:21.097745 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e9db9-8fe4-416a-87d6-de38ef45f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:21.097745 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e9db9-8fe4-416a-87d6-de38ef45f9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:21.120095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a74e9db9-8fe4-416a-87d6-de38ef45f9d0 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:21.121330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a74e9db9-8fe4-416a-87d6-de38ef45f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:21.195873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1114/2240] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:24:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:24:21.238457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf9f5f-086e-4ca7-88d2-9073caf99026 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:21.239939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-47bf9f5f-086e-4ca7-88d2-9073caf99026 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:21.314170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf9f5f-086e-4ca7-88d2-9073caf99026 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:21.320298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2f8e6-dc6c-4221-ac97-64c1463f1bef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:21.322296 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2f8e6-dc6c-4221-ac97-64c1463f1bef service swift] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:21.322788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2f8e6-dc6c-4221-ac97-64c1463f1bef service swift] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:21.322788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2f8e6-dc6c-4221-ac97-64c1463f1bef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:21.331708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2f8e6-dc6c-4221-ac97-64c1463f1bef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:21.333304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2f8e6-dc6c-4221-ac97-64c1463f1bef service swift] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:21.344902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1127/2241] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:24:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:24:22.465113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e9db9-8fe4-416a-87d6-de38ef45f9d0 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:22.604082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e9db9-8fe4-416a-87d6-de38ef45f9d0 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:22.610010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:22.610769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:22.610911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:22.611168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] PATH_INFO: `/v3/users/e2d0c91f08d1415fab87984d8a5ac339` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:22.612289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d0c91f08d1415fab87984d8a5ac339)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:22.614657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:22.683052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d0c91f08d1415fab87984d8a5ac339'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:22.683738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d0c91f08d1415fab87984d8a5ac339'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:22.694488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d0c91f08d1415fab87984d8a5ac339'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:22.724688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d0c91f08d1415fab87984d8a5ac339'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:22.725405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d0c91f08d1415fab87984d8a5ac339'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:22.750547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1115/2242] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:22 2022] DELETE /identity/v3/users/e2d0c91f08d1415fab87984d8a5ac339 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:22.754850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba752ac-e8cf-4670-a8b5-08d9642bb530 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:22.756241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba752ac-e8cf-4670-a8b5-08d9642bb530 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:22.756521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba752ac-e8cf-4670-a8b5-08d9642bb530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:22.784869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-4ba752ac-e8cf-4670-a8b5-08d9642bb530 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:22.786122 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4ba752ac-e8cf-4670-a8b5-08d9642bb530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:22.811710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:22.891474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1128/2243] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:22.898780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:22.900205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:22.946140 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:22.999344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada16bce-324c-4c81-bae6-4031bd0427ee admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:23.004257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b6360-744b-4196-bc56-b46fa19b4ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:23.004883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6360-744b-4196-bc56-b46fa19b4ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:23.005168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6360-744b-4196-bc56-b46fa19b4ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:23.005441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b6360-744b-4196-bc56-b46fa19b4ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:23.022169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b6360-744b-4196-bc56-b46fa19b4ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:23.022169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b6360-744b-4196-bc56-b46fa19b4ed8 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:23.030319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1116/2244] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:23.360905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba752ac-e8cf-4670-a8b5-08d9642bb530 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:23.362368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4ba752ac-e8cf-4670-a8b5-08d9642bb530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:23.386091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba752ac-e8cf-4670-a8b5-08d9642bb530 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:23.393944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:23.394875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:23.395343 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:23.395878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] PATH_INFO: `/v3/projects/12092a09a40c4d8b8bd6e857d981fa84` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:23.397361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=12092a09a40c4d8b8bd6e857d981fa84)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:23.399601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:23.457165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12092a09a40c4d8b8bd6e857d981fa84', 'request_id': 'req-ca485846-528b-4fce-a33b-ec56bcc1a6aa'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.466770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12092a09a40c4d8b8bd6e857d981fa84', 'request_id': 'req-ca485846-528b-4fce-a33b-ec56bcc1a6aa'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.474564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12092a09a40c4d8b8bd6e857d981fa84', 'request_id': 'req-ca485846-528b-4fce-a33b-ec56bcc1a6aa'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.492004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] The token cache is being invalidate because project 12092a09a40c4d8b8bd6e857d981fa84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:23.492288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.492935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.494469 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1129/2245] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:23 2022] DELETE /identity/v3/projects/12092a09a40c4d8b8bd6e857d981fa84 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:23.498920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b6360-744b-4196-bc56-b46fa19b4ed8 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:23.508484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b6360-744b-4196-bc56-b46fa19b4ed8 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:23.564169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:23.564885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:23.565252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:23.565758 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] PATH_INFO: `/v3/users/c50489b9a13044efa104762afedb0205` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:23.567486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c50489b9a13044efa104762afedb0205)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:23.569680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:23.616188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c50489b9a13044efa104762afedb0205'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.616836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c50489b9a13044efa104762afedb0205'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.623832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c50489b9a13044efa104762afedb0205'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.631072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c50489b9a13044efa104762afedb0205'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.631757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-849fd67d-7511-46fc-9f2c-a19261e0c8c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c50489b9a13044efa104762afedb0205'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:23.650030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1117/2246] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:23 2022] DELETE /identity/v3/users/c50489b9a13044efa104762afedb0205 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:24.082095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:24.084536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:24.111420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca485846-528b-4fce-a33b-ec56bcc1a6aa admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:24.232776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:24.233612 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:24.236568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:24.237091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] PATH_INFO: `/v3/projects/553cb799de5f4cf3a170e778c343ba50` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:24.239041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=553cb799de5f4cf3a170e778c343ba50)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:24.242102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:24.327687 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:24.332515 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553cb799de5f4cf3a170e778c343ba50', 'request_id': 'req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:24.346707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553cb799de5f4cf3a170e778c343ba50', 'request_id': 'req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:24.353294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553cb799de5f4cf3a170e778c343ba50', 'request_id': 'req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:24.371543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] The token cache is being invalidate because project 553cb799de5f4cf3a170e778c343ba50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:24.372007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:24.372781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-f19cf810-17e5-4ead-b5b9-9af4f4c4b292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:24.375228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1130/2247] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:24 2022] DELETE /identity/v3/projects/553cb799de5f4cf3a170e778c343ba50 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:25.609650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25c01b-11d1-40b4-8166-24e6271278b6 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:25.610153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25c01b-11d1-40b4-8166-24e6271278b6 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:25.610259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25c01b-11d1-40b4-8166-24e6271278b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:25.633767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-0c25c01b-11d1-40b4-8166-24e6271278b6 None None] MFA Rules not processed for user `1301a78c1a684c94acb9f26b56f39f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:25.635296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0c25c01b-11d1-40b4-8166-24e6271278b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:25.687296 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:25.707254 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1118/2248] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:34.368813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:35.703105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:37.412352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692023ef-6f72-4644-85d2-ef7b66a8c527 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:37.412999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692023ef-6f72-4644-85d2-ef7b66a8c527 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:37.412999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692023ef-6f72-4644-85d2-ef7b66a8c527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:37.444222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-692023ef-6f72-4644-85d2-ef7b66a8c527 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:37.445951 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-692023ef-6f72-4644-85d2-ef7b66a8c527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:37.549285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1131/2249] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:24:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:37.564193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25c01b-11d1-40b4-8166-24e6271278b6 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:37.565664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0c25c01b-11d1-40b4-8166-24e6271278b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:37.713689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25c01b-11d1-40b4-8166-24e6271278b6 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:37.735119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad778d-e154-44eb-9d9e-35276399fb3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:37.735907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad778d-e154-44eb-9d9e-35276399fb3e service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:37.736896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad778d-e154-44eb-9d9e-35276399fb3e service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:37.737461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad778d-e154-44eb-9d9e-35276399fb3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:37.756126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad778d-e154-44eb-9d9e-35276399fb3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:37.757947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad778d-e154-44eb-9d9e-35276399fb3e service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:37.771669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1119/2250] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:24:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:24:40.717121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27200d3c-6b35-452f-86f5-4547b473caea None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:40.717643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27200d3c-6b35-452f-86f5-4547b473caea None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:40.717764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27200d3c-6b35-452f-86f5-4547b473caea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:40.744013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-27200d3c-6b35-452f-86f5-4547b473caea None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:40.745282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-27200d3c-6b35-452f-86f5-4547b473caea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:40.757695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1132/2251] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 20:24:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:40.772415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad778d-e154-44eb-9d9e-35276399fb3e service swift] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:40.787162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad778d-e154-44eb-9d9e-35276399fb3e service swift] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:40.795733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10616b7-099c-48e1-b2d2-1b1707ebcc9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:40.796591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10616b7-099c-48e1-b2d2-1b1707ebcc9b service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:40.797090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10616b7-099c-48e1-b2d2-1b1707ebcc9b service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:40.797419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10616b7-099c-48e1-b2d2-1b1707ebcc9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:40.808914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10616b7-099c-48e1-b2d2-1b1707ebcc9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:40.812504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10616b7-099c-48e1-b2d2-1b1707ebcc9b service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:40.829411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1120/2252] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:24:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:24:43.709495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27200d3c-6b35-452f-86f5-4547b473caea None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:43.711531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-27200d3c-6b35-452f-86f5-4547b473caea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:43.820028 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27200d3c-6b35-452f-86f5-4547b473caea None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:43.826298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:43.826955 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:43.827237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:43.828294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] PATH_INFO: `/v3/users/1b8c41cfa6854a528f43e82e0fb26220` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:43.830117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8c41cfa6854a528f43e82e0fb26220)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:43.835605 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:43.907519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8c41cfa6854a528f43e82e0fb26220'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:43.908240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8c41cfa6854a528f43e82e0fb26220'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:43.925017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8c41cfa6854a528f43e82e0fb26220'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:43.926231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8c41cfa6854a528f43e82e0fb26220'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:43.953669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4a0b026a-393f-481c-8f1a-3974ec9efed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8c41cfa6854a528f43e82e0fb26220'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:43.993784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1133/2253] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:43 2022] DELETE /identity/v3/users/1b8c41cfa6854a528f43e82e0fb26220 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:44.328695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10616b7-099c-48e1-b2d2-1b1707ebcc9b service swift] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:44.366585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10616b7-099c-48e1-b2d2-1b1707ebcc9b service swift] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:44.465588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:44.466182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:44.466459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:44.466735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] PATH_INFO: `/v3/projects/25c983389ecf447c9ca4bd6ca01b5a32` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:44.467945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c983389ecf447c9ca4bd6ca01b5a32)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:44.469944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:44.540623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c983389ecf447c9ca4bd6ca01b5a32', 'request_id': 'req-e24625a1-f4ca-40ec-a600-136761e9902d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:44.550940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c983389ecf447c9ca4bd6ca01b5a32', 'request_id': 'req-e24625a1-f4ca-40ec-a600-136761e9902d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:44.558099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c983389ecf447c9ca4bd6ca01b5a32', 'request_id': 'req-e24625a1-f4ca-40ec-a600-136761e9902d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:44.571108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] The token cache is being invalidate because project 25c983389ecf447c9ca4bd6ca01b5a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:44.571398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:44.572647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e24625a1-f4ca-40ec-a600-136761e9902d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:44.574839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1121/2254] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:44 2022] DELETE /identity/v3/projects/25c983389ecf447c9ca4bd6ca01b5a32 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:44.581964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a63ae-25c8-4ea6-925d-e5dd62e5d443 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:44.582310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a63ae-25c8-4ea6-925d-e5dd62e5d443 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:44.582526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a63ae-25c8-4ea6-925d-e5dd62e5d443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:44.615367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5f2a63ae-25c8-4ea6-925d-e5dd62e5d443 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:44.616634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5f2a63ae-25c8-4ea6-925d-e5dd62e5d443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:44.636954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:44.745703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1134/2255] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:44.778493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c96b4-fff7-4575-96ef-2480ea4877a5 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:44.779127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c96b4-fff7-4575-96ef-2480ea4877a5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:44.779606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c96b4-fff7-4575-96ef-2480ea4877a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:44.798636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:44.807724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-bb9c96b4-fff7-4575-96ef-2480ea4877a5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:44.809313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-bb9c96b4-fff7-4575-96ef-2480ea4877a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:44.828570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1122/2256] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:44.841560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a63ae-25c8-4ea6-925d-e5dd62e5d443 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:44.848155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a63ae-25c8-4ea6-925d-e5dd62e5d443 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:44.854146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795e28a-fb28-41fc-b5a5-e332dd858e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:44.854773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795e28a-fb28-41fc-b5a5-e332dd858e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:44.855158 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795e28a-fb28-41fc-b5a5-e332dd858e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:44.855437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795e28a-fb28-41fc-b5a5-e332dd858e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:44.856121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795e28a-fb28-41fc-b5a5-e332dd858e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:24:44.856399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795e28a-fb28-41fc-b5a5-e332dd858e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:44.858125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795e28a-fb28-41fc-b5a5-e332dd858e16 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:44.876671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1135/2257] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:24:44.877263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c96b4-fff7-4575-96ef-2480ea4877a5 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:44.887054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c96b4-fff7-4575-96ef-2480ea4877a5 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:44.892956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83477d1c-92c4-4135-aca2-24805d23c3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:44.893650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83477d1c-92c4-4135-aca2-24805d23c3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:44.894056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83477d1c-92c4-4135-aca2-24805d23c3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:44.894326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83477d1c-92c4-4135-aca2-24805d23c3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:44.894955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83477d1c-92c4-4135-aca2-24805d23c3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:44.897101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83477d1c-92c4-4135-aca2-24805d23c3a5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:44.943220 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1123/2258] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:24:44 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:24:44.946352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795e28a-fb28-41fc-b5a5-e332dd858e16 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:44.964793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795e28a-fb28-41fc-b5a5-e332dd858e16 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:45.040792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e93d-1faf-4d54-895e-db5c8a7a2af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:45.041624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e93d-1faf-4d54-895e-db5c8a7a2af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:45.041904 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e93d-1faf-4d54-895e-db5c8a7a2af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:45.042174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e93d-1faf-4d54-895e-db5c8a7a2af6 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:45.042824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87e93d-1faf-4d54-895e-db5c8a7a2af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:45.044787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87e93d-1faf-4d54-895e-db5c8a7a2af6 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:45.069085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1136/2259] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:24:44 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:24:45.073136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83477d1c-92c4-4135-aca2-24805d23c3a5 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:45.085554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83477d1c-92c4-4135-aca2-24805d23c3a5 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:45.090344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17529941-4194-4a90-a770-d1d854382fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:45.090982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17529941-4194-4a90-a770-d1d854382fbc admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:45.091257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17529941-4194-4a90-a770-d1d854382fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:45.091525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17529941-4194-4a90-a770-d1d854382fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:45.092172 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17529941-4194-4a90-a770-d1d854382fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:45.094190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17529941-4194-4a90-a770-d1d854382fbc admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:45.108638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1124/2260] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:45.112549 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e93d-1faf-4d54-895e-db5c8a7a2af6 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:45.126889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e93d-1faf-4d54-895e-db5c8a7a2af6 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:45.131818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea9772-05c6-4f68-ba2c-bdbd5672ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:45.132434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea9772-05c6-4f68-ba2c-bdbd5672ca32 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:45.132709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea9772-05c6-4f68-ba2c-bdbd5672ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:45.132975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea9772-05c6-4f68-ba2c-bdbd5672ca32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:45.133654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea9772-05c6-4f68-ba2c-bdbd5672ca32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:45.135537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea9772-05c6-4f68-ba2c-bdbd5672ca32 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:45.147969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1137/2261] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:24:45.152180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17529941-4194-4a90-a770-d1d854382fbc admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:45.167190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17529941-4194-4a90-a770-d1d854382fbc admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:45.172890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c75f22-656a-4eeb-a402-92424f85560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:45.173574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c75f22-656a-4eeb-a402-92424f85560c admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:45.173857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c75f22-656a-4eeb-a402-92424f85560c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:45.174147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c75f22-656a-4eeb-a402-92424f85560c admin admin] PATH_INFO: `/v3/projects/87881339022b40ba9b9ef1e03c62d93d/users/3636404dfdfa4a0e916abf8831b8dc14/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:45.194133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c75f22-656a-4eeb-a402-92424f85560c admin admin] RBAC: Authorizing `identity:create_grant(project_id=87881339022b40ba9b9ef1e03c62d93d, user_id=3636404dfdfa4a0e916abf8831b8dc14, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:45.197193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c75f22-656a-4eeb-a402-92424f85560c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:45.211203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1125/2262] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:24:45 2022] PUT /identity/v3/projects/87881339022b40ba9b9ef1e03c62d93d/users/3636404dfdfa4a0e916abf8831b8dc14/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:45.221888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38a2ad-d0c8-4087-b49f-028e3ca30568 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:45.222374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38a2ad-d0c8-4087-b49f-028e3ca30568 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:45.222649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38a2ad-d0c8-4087-b49f-028e3ca30568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:45.258329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-fb38a2ad-d0c8-4087-b49f-028e3ca30568 None None] MFA Rules not processed for user `3636404dfdfa4a0e916abf8831b8dc14`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:45.259982 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fb38a2ad-d0c8-4087-b49f-028e3ca30568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:45.359365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1138/2263] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:45.368314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c75f22-656a-4eeb-a402-92424f85560c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:45.370762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f1c75f22-656a-4eeb-a402-92424f85560c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:45.458852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c75f22-656a-4eeb-a402-92424f85560c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:45.462389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477adb40-cbec-44f7-b269-321ff62bcda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:45.462989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477adb40-cbec-44f7-b269-321ff62bcda3 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:45.463262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477adb40-cbec-44f7-b269-321ff62bcda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:45.463535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477adb40-cbec-44f7-b269-321ff62bcda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:45.473766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477adb40-cbec-44f7-b269-321ff62bcda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:45.475360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477adb40-cbec-44f7-b269-321ff62bcda3 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:45.489695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1126/2264] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:55.352490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:55.482179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:56.377568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38a2ad-d0c8-4087-b49f-028e3ca30568 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:56.439092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38a2ad-d0c8-4087-b49f-028e3ca30568 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:56.443781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:56.444409 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:56.444683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:56.444962 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] PATH_INFO: `/v3/users/3636404dfdfa4a0e916abf8831b8dc14` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:56.446086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3636404dfdfa4a0e916abf8831b8dc14)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:56.448024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:56.494538 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3636404dfdfa4a0e916abf8831b8dc14'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:56.495176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3636404dfdfa4a0e916abf8831b8dc14'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:56.501939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3636404dfdfa4a0e916abf8831b8dc14'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:56.502557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3636404dfdfa4a0e916abf8831b8dc14'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:56.509806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3636404dfdfa4a0e916abf8831b8dc14'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:56.527689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1139/2265] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:24:56 2022] DELETE /identity/v3/users/3636404dfdfa4a0e916abf8831b8dc14 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:56.532608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df0017-0ee0-464c-9ff3-da70d74a8e95 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:56.533049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df0017-0ee0-464c-9ff3-da70d74a8e95 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:56.533365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df0017-0ee0-464c-9ff3-da70d74a8e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:56.552928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e1df0017-0ee0-464c-9ff3-da70d74a8e95 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:56.554280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e1df0017-0ee0-464c-9ff3-da70d74a8e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:56.616237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1127/2266] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:56.627362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:56.629208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:56.690789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b824-928d-4fd9-aa2f-d6298fe7a67a admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:56.694667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49937a6d-e164-44d6-9b76-82b6b86b2ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:56.695268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49937a6d-e164-44d6-9b76-82b6b86b2ffe service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:56.695544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49937a6d-e164-44d6-9b76-82b6b86b2ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:56.695814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49937a6d-e164-44d6-9b76-82b6b86b2ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:56.705651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49937a6d-e164-44d6-9b76-82b6b86b2ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:56.707071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49937a6d-e164-44d6-9b76-82b6b86b2ffe service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:56.717276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1140/2267] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:57.014974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df0017-0ee0-464c-9ff3-da70d74a8e95 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:57.016683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e1df0017-0ee0-464c-9ff3-da70d74a8e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:57.030693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df0017-0ee0-464c-9ff3-da70d74a8e95 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:57.043394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:57.046264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.046565 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.046814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] PATH_INFO: `/v3/projects/87881339022b40ba9b9ef1e03c62d93d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.048023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87881339022b40ba9b9ef1e03c62d93d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:57.050207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:57.102286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87881339022b40ba9b9ef1e03c62d93d', 'request_id': 'req-144d0669-f113-4389-a542-7f9b38b036b2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:57.110929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87881339022b40ba9b9ef1e03c62d93d', 'request_id': 'req-144d0669-f113-4389-a542-7f9b38b036b2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:57.118723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87881339022b40ba9b9ef1e03c62d93d', 'request_id': 'req-144d0669-f113-4389-a542-7f9b38b036b2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:57.131040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] The token cache is being invalidate because project 87881339022b40ba9b9ef1e03c62d93d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:24:57.131326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:57.131996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-144d0669-f113-4389-a542-7f9b38b036b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:24:57.138200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1128/2268] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:24:57 2022] DELETE /identity/v3/projects/87881339022b40ba9b9ef1e03c62d93d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:24:57.357760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fda9-b719-44fe-9299-9e00d83b5e65 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.358072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fda9-b719-44fe-9299-9e00d83b5e65 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.358348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fda9-b719-44fe-9299-9e00d83b5e65 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.359716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1141/2269] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:24:57.365417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60d673-7f91-42e4-bb08-e26af720e628 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.365732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60d673-7f91-42e4-bb08-e26af720e628 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.366004 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60d673-7f91-42e4-bb08-e26af720e628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.388059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4e60d673-7f91-42e4-bb08-e26af720e628 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:57.389202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4e60d673-7f91-42e4-bb08-e26af720e628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:57.457928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1129/2270] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:24:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:57.469993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fda9-b719-44fe-9299-9e00d83b5e65 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:57.471831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6c12fda9-b719-44fe-9299-9e00d83b5e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:57.510771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:57.565133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fda9-b719-44fe-9299-9e00d83b5e65 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:57.568882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47c7ea-974c-4165-8b55-9a5e1957eab9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bafea2367e4c7992354bb2c23e7d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:57.569474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47c7ea-974c-4165-8b55-9a5e1957eab9 service glance] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.569774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47c7ea-974c-4165-8b55-9a5e1957eab9 service glance] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.570041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47c7ea-974c-4165-8b55-9a5e1957eab9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.580330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47c7ea-974c-4165-8b55-9a5e1957eab9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:57.581776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47c7ea-974c-4165-8b55-9a5e1957eab9 service glance] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:57.592014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1142/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:57.661473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35153b17-fc0e-42ba-b5ad-f8e93d6468c5 None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.662017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35153b17-fc0e-42ba-b5ad-f8e93d6468c5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.662017 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35153b17-fc0e-42ba-b5ad-f8e93d6468c5 None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.663382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1130/2272] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:24:57.668003 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec233d-54db-4da1-9be9-3dad049ef596 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.668290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec233d-54db-4da1-9be9-3dad049ef596 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.668565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec233d-54db-4da1-9be9-3dad049ef596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.688214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c0ec233d-54db-4da1-9be9-3dad049ef596 None None] MFA Rules not processed for user `8f5a88f263014f15a6f59cfdaa2bb6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:57.689404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c0ec233d-54db-4da1-9be9-3dad049ef596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:57.738490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1143/2273] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:24:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:24:57.750660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35153b17-fc0e-42ba-b5ad-f8e93d6468c5 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:57.752367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-35153b17-fc0e-42ba-b5ad-f8e93d6468c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:57.765999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35153b17-fc0e-42ba-b5ad-f8e93d6468c5 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:57.769444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e6dfb-272a-4bda-bd98-b1718aa4b3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:57.770079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e6dfb-272a-4bda-bd98-b1718aa4b3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.770341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e6dfb-272a-4bda-bd98-b1718aa4b3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.770609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e6dfb-272a-4bda-bd98-b1718aa4b3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.777739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e6dfb-272a-4bda-bd98-b1718aa4b3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:57.779723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e6dfb-272a-4bda-bd98-b1718aa4b3c0 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:57.787535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1131/2274] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:57.869414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9669d-e9ca-47e1-8a2b-e45670412249 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.869801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9669d-e9ca-47e1-8a2b-e45670412249 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.870011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9669d-e9ca-47e1-8a2b-e45670412249 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.871391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1144/2275] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:24:57.876393 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddfae0-7541-469b-9faa-b3beb45b46df None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.876681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddfae0-7541-469b-9faa-b3beb45b46df None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.876958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddfae0-7541-469b-9faa-b3beb45b46df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.884655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:24:57.898645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-99ddfae0-7541-469b-9faa-b3beb45b46df None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:57.899844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-99ddfae0-7541-469b-9faa-b3beb45b46df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:57.909645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1132/2276] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:24:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:24:57.920021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9669d-e9ca-47e1-8a2b-e45670412249 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:24:57.931057 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9669d-e9ca-47e1-8a2b-e45670412249 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:24:57.934576 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d9501-21a6-4a48-9bfb-59fb80b9284d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:24:57.935171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d9501-21a6-4a48-9bfb-59fb80b9284d service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:57.935445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d9501-21a6-4a48-9bfb-59fb80b9284d service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:57.935710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d9501-21a6-4a48-9bfb-59fb80b9284d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:57.946108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d9501-21a6-4a48-9bfb-59fb80b9284d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:24:57.947531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d9501-21a6-4a48-9bfb-59fb80b9284d service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:24:57.958676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1145/2277] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:24:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:24:59.074204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a15941-c462-4615-8f06-47d7d23b9988 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:24:59.074728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a15941-c462-4615-8f06-47d7d23b9988 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:24:59.074885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a15941-c462-4615-8f06-47d7d23b9988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:24:59.095314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-b5a15941-c462-4615-8f06-47d7d23b9988 None None] MFA Rules not processed for user `776c647ef6d34d989ac12d4e91cc62ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:24:59.096625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b5a15941-c462-4615-8f06-47d7d23b9988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:24:59.155496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1133/2278] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:24:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:25:02.920529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a678df-8595-40f0-8024-e8d7b0c21556 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:25:02.921025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a678df-8595-40f0-8024-e8d7b0c21556 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:25:02.921115 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a678df-8595-40f0-8024-e8d7b0c21556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:25:02.940852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-32a678df-8595-40f0-8024-e8d7b0c21556 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:25:02.942360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-32a678df-8595-40f0-8024-e8d7b0c21556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:25:03.000727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1146/2279] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:25:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:25:03.015312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a15941-c462-4615-8f06-47d7d23b9988 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:25:03.017015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b5a15941-c462-4615-8f06-47d7d23b9988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:25:03.088028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a15941-c462-4615-8f06-47d7d23b9988 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:25:03.094331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593dcd1-61a3-4112-95bd-a820322928dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:25:03.094954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593dcd1-61a3-4112-95bd-a820322928dc service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:25:03.095231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593dcd1-61a3-4112-95bd-a820322928dc service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:25:03.095501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593dcd1-61a3-4112-95bd-a820322928dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:25:03.104808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593dcd1-61a3-4112-95bd-a820322928dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:25:03.107105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593dcd1-61a3-4112-95bd-a820322928dc service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:25:03.116177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1134/2280] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:25:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:25:12.992568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=79 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:25:13.112797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=59 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:25:25.839556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957eedc-a831-4e1f-8688-7ec922a2df8c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:25:25.840130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957eedc-a831-4e1f-8688-7ec922a2df8c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:25:25.840233 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957eedc-a831-4e1f-8688-7ec922a2df8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:25:25.859214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-4957eedc-a831-4e1f-8688-7ec922a2df8c None None] MFA Rules not processed for user `10d2803e109849fda2828f66f51b8a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:25:25.860412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4957eedc-a831-4e1f-8688-7ec922a2df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:25:25.915893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1147/2281] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:25:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:25:27.029893 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d57e-631d-40a1-9cfe-b93d2a198b5e None None] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:25:27.030429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d57e-631d-40a1-9cfe-b93d2a198b5e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:25:27.030429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4d57e-631d-40a1-9cfe-b93d2a198b5e None None] PATH_INFO: `/` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:25:27.031781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1135/2282] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 20:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:25:27.036818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a37ba-b33d-4c2c-866d-358f8ed87305 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:25:27.037221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a37ba-b33d-4c2c-866d-358f8ed87305 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:25:27.037359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a37ba-b33d-4c2c-866d-358f8ed87305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:25:27.057158 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ed6a37ba-b33d-4c2c-866d-358f8ed87305 None None] MFA Rules not processed for user `5181c4b31ac240469aff7a0aee35eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:25:27.058418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ed6a37ba-b33d-4c2c-866d-358f8ed87305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:25:27.118841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1148/2283] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 20:25:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:25:27.126340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4d57e-631d-40a1-9cfe-b93d2a198b5e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:25:27.139824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4d57e-631d-40a1-9cfe-b93d2a198b5e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:25:27.144704 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5b6c5-3b77-48be-bf02-1637674b7bb8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:25:27.145282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5b6c5-3b77-48be-bf02-1637674b7bb8 service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:25:27.145586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5b6c5-3b77-48be-bf02-1637674b7bb8 service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:25:27.145857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5b6c5-3b77-48be-bf02-1637674b7bb8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:25:27.147517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c6f5b6c5-3b77-48be-bf02-1637674b7bb8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:25:27.159200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5b6c5-3b77-48be-bf02-1637674b7bb8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:25:27.160805 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5b6c5-3b77-48be-bf02-1637674b7bb8 service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:25:27.170606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1136/2284] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:25:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:25:29.016903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd1f71-ebc5-40c0-83c8-1d8a79994c3e None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:25:29.017305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd1f71-ebc5-40c0-83c8-1d8a79994c3e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:25:29.017493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd1f71-ebc5-40c0-83c8-1d8a79994c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:25:29.038137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f7bd1f71-ebc5-40c0-83c8-1d8a79994c3e None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:25:29.039355 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f7bd1f71-ebc5-40c0-83c8-1d8a79994c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:25:29.047874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1149/2285] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 20:25:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:25:29.071343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5b6c5-3b77-48be-bf02-1637674b7bb8 service placement] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:25:29.082841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5b6c5-3b77-48be-bf02-1637674b7bb8 service placement] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:25:29.092698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188614d0-34db-4616-9443-b92739bf1537 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:25:29.093315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188614d0-34db-4616-9443-b92739bf1537 service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:25:29.093634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188614d0-34db-4616-9443-b92739bf1537 service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:25:29.093911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188614d0-34db-4616-9443-b92739bf1537 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:25:29.102538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188614d0-34db-4616-9443-b92739bf1537 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:25:29.104109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188614d0-34db-4616-9443-b92739bf1537 service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:25:29.113284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1137/2286] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:25:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:25:39.033031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=57 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:25:39.109659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=12 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:13.286949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3dea3-28bc-482d-aa22-d5f4e9f45d61 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:13.287506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3dea3-28bc-482d-aa22-d5f4e9f45d61 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:13.287601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3dea3-28bc-482d-aa22-d5f4e9f45d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:13.308578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-72a3dea3-28bc-482d-aa22-d5f4e9f45d61 None None] MFA Rules not processed for user `1301a78c1a684c94acb9f26b56f39f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:13.309797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-72a3dea3-28bc-482d-aa22-d5f4e9f45d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:13.365977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1150/2287] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:14.567560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188614d0-34db-4616-9443-b92739bf1537 service swift] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:14.569058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-188614d0-34db-4616-9443-b92739bf1537 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:14.634110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188614d0-34db-4616-9443-b92739bf1537 service swift] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:14.638918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:14.639695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:14.639996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:14.640357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] PATH_INFO: `/v3/users/776c647ef6d34d989ac12d4e91cc62ab` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:14.641633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=776c647ef6d34d989ac12d4e91cc62ab)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:14.643551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:14.689373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '776c647ef6d34d989ac12d4e91cc62ab'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:14.690080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '776c647ef6d34d989ac12d4e91cc62ab'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:14.697307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '776c647ef6d34d989ac12d4e91cc62ab'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:14.704640 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '776c647ef6d34d989ac12d4e91cc62ab'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:14.705293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-efbc9eae-82d3-4eec-9364-941f216c6316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '776c647ef6d34d989ac12d4e91cc62ab'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:14.722149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1138/2288] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:26:14 2022] DELETE /identity/v3/users/776c647ef6d34d989ac12d4e91cc62ab => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:26:14.957039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3dea3-28bc-482d-aa22-d5f4e9f45d61 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:14.968004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3dea3-28bc-482d-aa22-d5f4e9f45d61 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:15.033719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:15.034278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.034556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.034830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] PATH_INFO: `/v3/projects/1c2141e33c264250aa7642f46a9b4048` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.035929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2141e33c264250aa7642f46a9b4048)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:15.037892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:15.083224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2141e33c264250aa7642f46a9b4048', 'request_id': 'req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:15.090108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2141e33c264250aa7642f46a9b4048', 'request_id': 'req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:15.096185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2141e33c264250aa7642f46a9b4048', 'request_id': 'req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:15.106286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] The token cache is being invalidate because project 1c2141e33c264250aa7642f46a9b4048 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:26:15.106568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:15.107187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fce4e6fa-91e3-4904-8bb0-8d504d8ef83b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:15.108636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1151/2289] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:26:14 2022] DELETE /identity/v3/projects/1c2141e33c264250aa7642f46a9b4048 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:26:15.113417 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c778c-dde1-4ea4-a942-3a162e9ba0ab None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.113723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c778c-dde1-4ea4-a942-3a162e9ba0ab None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.113996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c778c-dde1-4ea4-a942-3a162e9ba0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.133575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ae4c778c-dde1-4ea4-a942-3a162e9ba0ab None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:15.134773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ae4c778c-dde1-4ea4-a942-3a162e9ba0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:15.192549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1139/2290] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:15.200266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7a1b-6152-46c1-a4ed-fe78df7a184d None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.200586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7a1b-6152-46c1-a4ed-fe78df7a184d None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.200857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7a1b-6152-46c1-a4ed-fe78df7a184d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.220697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-eafd7a1b-6152-46c1-a4ed-fe78df7a184d None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:15.221927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-eafd7a1b-6152-46c1-a4ed-fe78df7a184d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:15.229743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1152/2291] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:15.233661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c778c-dde1-4ea4-a942-3a162e9ba0ab None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:15.243174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c778c-dde1-4ea4-a942-3a162e9ba0ab None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:15.247852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be262652-3430-45b0-bfc8-62c74c4ef367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:15.248453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be262652-3430-45b0-bfc8-62c74c4ef367 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.248731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be262652-3430-45b0-bfc8-62c74c4ef367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.249002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be262652-3430-45b0-bfc8-62c74c4ef367 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.249685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be262652-3430-45b0-bfc8-62c74c4ef367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:26:15.250536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be262652-3430-45b0-bfc8-62c74c4ef367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:15.252128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be262652-3430-45b0-bfc8-62c74c4ef367 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:15.263596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1140/2292] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:26:15.268470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7a1b-6152-46c1-a4ed-fe78df7a184d None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:15.277398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7a1b-6152-46c1-a4ed-fe78df7a184d None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:15.282053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df75a6-230e-4672-b953-8a2d8816bb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:15.282661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df75a6-230e-4672-b953-8a2d8816bb74 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.282915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df75a6-230e-4672-b953-8a2d8816bb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.283184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df75a6-230e-4672-b953-8a2d8816bb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.283869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df75a6-230e-4672-b953-8a2d8816bb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:15.285741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df75a6-230e-4672-b953-8a2d8816bb74 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:15.311565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1153/2293] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:26:15 2022] POST /identity/v3/projects => generated 399 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:26:15.316139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be262652-3430-45b0-bfc8-62c74c4ef367 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:15.326250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be262652-3430-45b0-bfc8-62c74c4ef367 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:15.330352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:15.386044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc82c18-2c52-42a9-a4d8-b7a33d630e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:15.386649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc82c18-2c52-42a9-a4d8-b7a33d630e73 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.386929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc82c18-2c52-42a9-a4d8-b7a33d630e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.387200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc82c18-2c52-42a9-a4d8-b7a33d630e73 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.387822 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc82c18-2c52-42a9-a4d8-b7a33d630e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:15.389751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc82c18-2c52-42a9-a4d8-b7a33d630e73 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:15.412192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1141/2294] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:26:15 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:26:15.416329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df75a6-230e-4672-b953-8a2d8816bb74 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:15.426429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df75a6-230e-4672-b953-8a2d8816bb74 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:15.430997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf383f5e-c2fd-4f56-b7d2-a68bb9a8ae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:15.431596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf383f5e-c2fd-4f56-b7d2-a68bb9a8ae57 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.432331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf383f5e-c2fd-4f56-b7d2-a68bb9a8ae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.432331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf383f5e-c2fd-4f56-b7d2-a68bb9a8ae57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.432760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf383f5e-c2fd-4f56-b7d2-a68bb9a8ae57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:15.434396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf383f5e-c2fd-4f56-b7d2-a68bb9a8ae57 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:15.435714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:15.447065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1154/2295] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:26:15.451209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc82c18-2c52-42a9-a4d8-b7a33d630e73 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:15.461631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc82c18-2c52-42a9-a4d8-b7a33d630e73 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:15.466403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e6615-1141-4707-bcc3-25a5a21d259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:15.467683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e6615-1141-4707-bcc3-25a5a21d259e admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.468105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e6615-1141-4707-bcc3-25a5a21d259e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.468401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e6615-1141-4707-bcc3-25a5a21d259e admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.469700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e6615-1141-4707-bcc3-25a5a21d259e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:15.471308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e6615-1141-4707-bcc3-25a5a21d259e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:15.482340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1142/2296] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:26:15.487099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf383f5e-c2fd-4f56-b7d2-a68bb9a8ae57 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:15.497176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf383f5e-c2fd-4f56-b7d2-a68bb9a8ae57 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:15.502402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62746345-049c-4375-849f-e5ba816236e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:15.503044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62746345-049c-4375-849f-e5ba816236e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.503322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62746345-049c-4375-849f-e5ba816236e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.503600 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62746345-049c-4375-849f-e5ba816236e8 admin admin] PATH_INFO: `/v3/projects/45f9210a3f7c49079f9003938acf1ec1/users/811f6cb7a5024e9ebcc920af6e8b6827/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.518402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62746345-049c-4375-849f-e5ba816236e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f9210a3f7c49079f9003938acf1ec1, user_id=811f6cb7a5024e9ebcc920af6e8b6827, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:15.521050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62746345-049c-4375-849f-e5ba816236e8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:15.533743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1155/2297] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:26:15 2022] PUT /identity/v3/projects/45f9210a3f7c49079f9003938acf1ec1/users/811f6cb7a5024e9ebcc920af6e8b6827/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:26:15.543990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc544c3-25a8-48dd-ac25-af3f2aa0a8bd None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.544216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc544c3-25a8-48dd-ac25-af3f2aa0a8bd None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.544488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc544c3-25a8-48dd-ac25-af3f2aa0a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.572940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-abc544c3-25a8-48dd-ac25-af3f2aa0a8bd None None] MFA Rules not processed for user `811f6cb7a5024e9ebcc920af6e8b6827`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:15.574210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-abc544c3-25a8-48dd-ac25-af3f2aa0a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:15.638073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1143/2298] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:15.649507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62746345-049c-4375-849f-e5ba816236e8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:15.651217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-62746345-049c-4375-849f-e5ba816236e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:15.707284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62746345-049c-4375-849f-e5ba816236e8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:15.710877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f968f-d4aa-4fe2-8f13-dc10c3de26b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:15.711474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f968f-d4aa-4fe2-8f13-dc10c3de26b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:15.711748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f968f-d4aa-4fe2-8f13-dc10c3de26b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:15.712016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f968f-d4aa-4fe2-8f13-dc10c3de26b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:15.720704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f968f-d4aa-4fe2-8f13-dc10c3de26b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:15.722150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f968f-d4aa-4fe2-8f13-dc10c3de26b3 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:15.730680 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1156/2299] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:26:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:26:22.813155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc544c3-25a8-48dd-ac25-af3f2aa0a8bd None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:22.876061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc544c3-25a8-48dd-ac25-af3f2aa0a8bd None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:22.880526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:22.881149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:22.881419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:22.881717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] PATH_INFO: `/v3/users/811f6cb7a5024e9ebcc920af6e8b6827` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:22.882818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=811f6cb7a5024e9ebcc920af6e8b6827)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:22.884739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:22.900192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:22.931241 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811f6cb7a5024e9ebcc920af6e8b6827'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:22.931824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811f6cb7a5024e9ebcc920af6e8b6827'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:22.939244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811f6cb7a5024e9ebcc920af6e8b6827'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:22.946573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811f6cb7a5024e9ebcc920af6e8b6827'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:22.947186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811f6cb7a5024e9ebcc920af6e8b6827'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:22.964420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1144/2300] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:26:22 2022] DELETE /identity/v3/users/811f6cb7a5024e9ebcc920af6e8b6827 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:26:22.968927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ec2ee-4402-4267-8364-cd1773a749a9 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:22.969439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ec2ee-4402-4267-8364-cd1773a749a9 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:22.969439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ec2ee-4402-4267-8364-cd1773a749a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:22.988290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ba8ec2ee-4402-4267-8364-cd1773a749a9 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:22.989466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ba8ec2ee-4402-4267-8364-cd1773a749a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:23.045889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1157/2301] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:23.055840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:23.057215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:23.113329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1dc88b-2d8e-4801-9880-06adbec759e7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:23.116915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a873a6-548e-428c-80e8-8f62a237e1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:23.117506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a873a6-548e-428c-80e8-8f62a237e1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.117802 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a873a6-548e-428c-80e8-8f62a237e1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.118068 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a873a6-548e-428c-80e8-8f62a237e1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:23.127321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a873a6-548e-428c-80e8-8f62a237e1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:23.128710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a873a6-548e-428c-80e8-8f62a237e1d3 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:23.137959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1145/2302] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:26:23.489447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ec2ee-4402-4267-8364-cd1773a749a9 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:23.491082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ba8ec2ee-4402-4267-8364-cd1773a749a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:23.506022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ec2ee-4402-4267-8364-cd1773a749a9 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:23.510718 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:23.511426 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.511772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.512111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] PATH_INFO: `/v3/projects/45f9210a3f7c49079f9003938acf1ec1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:23.513368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f9210a3f7c49079f9003938acf1ec1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:23.515359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:23.561778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f9210a3f7c49079f9003938acf1ec1', 'request_id': 'req-7c395f25-ad83-4a7b-baea-7cefe5536c4a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:23.569852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f9210a3f7c49079f9003938acf1ec1', 'request_id': 'req-7c395f25-ad83-4a7b-baea-7cefe5536c4a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:23.576855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f9210a3f7c49079f9003938acf1ec1', 'request_id': 'req-7c395f25-ad83-4a7b-baea-7cefe5536c4a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:23.583622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:23.588685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] The token cache is being invalidate because project 45f9210a3f7c49079f9003938acf1ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:26:23.589000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:23.589673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7c395f25-ad83-4a7b-baea-7cefe5536c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:26:23.595297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1158/2303] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:26:23 2022] DELETE /identity/v3/projects/45f9210a3f7c49079f9003938acf1ec1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:26:23.596922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983da7e-640a-45b4-ba18-931efa524a56 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.597201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983da7e-640a-45b4-ba18-931efa524a56 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.597473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983da7e-640a-45b4-ba18-931efa524a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:23.617234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-0983da7e-640a-45b4-ba18-931efa524a56 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:23.619095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0983da7e-640a-45b4-ba18-931efa524a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:23.685142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1146/2304] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:26:23.693220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688362f-ab78-4fdf-b785-7178cd6001f5 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.693529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688362f-ab78-4fdf-b785-7178cd6001f5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.693852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688362f-ab78-4fdf-b785-7178cd6001f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:23.715175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6688362f-ab78-4fdf-b785-7178cd6001f5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:23.716374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6688362f-ab78-4fdf-b785-7178cd6001f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:23.727340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1159/2305] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:23.730605 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983da7e-640a-45b4-ba18-931efa524a56 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:23.740337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983da7e-640a-45b4-ba18-931efa524a56 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:23.744788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa1d04-aba3-4d0f-850f-9d2335f96ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:23.745403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa1d04-aba3-4d0f-850f-9d2335f96ddb admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.745714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa1d04-aba3-4d0f-850f-9d2335f96ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.745981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa1d04-aba3-4d0f-850f-9d2335f96ddb admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:23.746616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa1d04-aba3-4d0f-850f-9d2335f96ddb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:26:23.746912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa1d04-aba3-4d0f-850f-9d2335f96ddb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:23.748481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa1d04-aba3-4d0f-850f-9d2335f96ddb admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:23.759909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1147/2306] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:26:23.764118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688362f-ab78-4fdf-b785-7178cd6001f5 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:23.774027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688362f-ab78-4fdf-b785-7178cd6001f5 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:23.778848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01797410-4d04-4cb4-a313-8c4c812d1dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:23.779445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01797410-4d04-4cb4-a313-8c4c812d1dae admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.779720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01797410-4d04-4cb4-a313-8c4c812d1dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.779989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01797410-4d04-4cb4-a313-8c4c812d1dae admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:23.780627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01797410-4d04-4cb4-a313-8c4c812d1dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:23.783171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01797410-4d04-4cb4-a313-8c4c812d1dae admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:23.809260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1160/2307] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:26:23 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:26:23.814027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa1d04-aba3-4d0f-850f-9d2335f96ddb admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:23.824265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa1d04-aba3-4d0f-850f-9d2335f96ddb admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:23.845361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:23.883124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb57f53-b7d7-457d-8520-6e5bcac0c5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:23.883733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb57f53-b7d7-457d-8520-6e5bcac0c5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.884010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb57f53-b7d7-457d-8520-6e5bcac0c5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.884279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb57f53-b7d7-457d-8520-6e5bcac0c5f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:23.884944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb57f53-b7d7-457d-8520-6e5bcac0c5f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:23.886878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb57f53-b7d7-457d-8520-6e5bcac0c5f6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:23.907529 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1148/2308] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:26:23 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:26:23.911636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01797410-4d04-4cb4-a313-8c4c812d1dae admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:23.922301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01797410-4d04-4cb4-a313-8c4c812d1dae admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:23.927050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b92c9-3862-4dea-b04d-1374e4e47590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:23.927646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b92c9-3862-4dea-b04d-1374e4e47590 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.927938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b92c9-3862-4dea-b04d-1374e4e47590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.928192 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b92c9-3862-4dea-b04d-1374e4e47590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:23.928827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b92c9-3862-4dea-b04d-1374e4e47590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:23.930447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b92c9-3862-4dea-b04d-1374e4e47590 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:23.942015 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1161/2309] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:26:23.946172 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb57f53-b7d7-457d-8520-6e5bcac0c5f6 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:23.956898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb57f53-b7d7-457d-8520-6e5bcac0c5f6 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:23.962391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e806c9-dc6a-45d2-99ab-b27074bd4143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:23.962997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e806c9-dc6a-45d2-99ab-b27074bd4143 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.963271 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e806c9-dc6a-45d2-99ab-b27074bd4143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.963537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e806c9-dc6a-45d2-99ab-b27074bd4143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:23.964171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e806c9-dc6a-45d2-99ab-b27074bd4143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:23.965797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e806c9-dc6a-45d2-99ab-b27074bd4143 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:23.976956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1149/2310] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:26:23.981266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b92c9-3862-4dea-b04d-1374e4e47590 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:23.992235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b92c9-3862-4dea-b04d-1374e4e47590 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:23.996719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996141-c4ca-4089-bcf2-1feae439952c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:23.997406 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996141-c4ca-4089-bcf2-1feae439952c admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:23.997702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996141-c4ca-4089-bcf2-1feae439952c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:23.998039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a996141-c4ca-4089-bcf2-1feae439952c admin admin] PATH_INFO: `/v3/projects/5d5f5bc40cdc4be792aff778d3020c6d/users/5500db61c95f4245945784ae6b13f197/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:24.012788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a996141-c4ca-4089-bcf2-1feae439952c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5f5bc40cdc4be792aff778d3020c6d, user_id=5500db61c95f4245945784ae6b13f197, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:24.015485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a996141-c4ca-4089-bcf2-1feae439952c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:24.031842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1162/2311] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:26:23 2022] PUT /identity/v3/projects/5d5f5bc40cdc4be792aff778d3020c6d/users/5500db61c95f4245945784ae6b13f197/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:26:24.037316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8de42-89d6-42eb-ba70-19b0d438736b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:24.037631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8de42-89d6-42eb-ba70-19b0d438736b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:24.037907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8de42-89d6-42eb-ba70-19b0d438736b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:24.057079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e9c8de42-89d6-42eb-ba70-19b0d438736b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:24.058320 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e9c8de42-89d6-42eb-ba70-19b0d438736b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:24.117536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1150/2312] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:24.127760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996141-c4ca-4089-bcf2-1feae439952c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:24.129638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9a996141-c4ca-4089-bcf2-1feae439952c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:24.190362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a996141-c4ca-4089-bcf2-1feae439952c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:24.194287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ffb6d-a11d-451d-b9f5-599bff48e90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:24.194908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ffb6d-a11d-451d-b9f5-599bff48e90f service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:24.195181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ffb6d-a11d-451d-b9f5-599bff48e90f service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:24.195445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ffb6d-a11d-451d-b9f5-599bff48e90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:24.205622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ffb6d-a11d-451d-b9f5-599bff48e90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:24.207080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ffb6d-a11d-451d-b9f5-599bff48e90f service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:24.216725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1163/2313] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:26:29.933820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477716ec-c78d-4a20-ad1d-c985840b7e6a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:29.935427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477716ec-c78d-4a20-ad1d-c985840b7e6a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:29.935778 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477716ec-c78d-4a20-ad1d-c985840b7e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:29.969622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d72c7-fab0-4545-b9d8-1f9012e098c1 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:29.970143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d72c7-fab0-4545-b9d8-1f9012e098c1 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:29.970143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d72c7-fab0-4545-b9d8-1f9012e098c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:29.989415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-477716ec-c78d-4a20-ad1d-c985840b7e6a None None] MFA Rules not processed for user `5500db61c95f4245945784ae6b13f197`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:29.991690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-477716ec-c78d-4a20-ad1d-c985840b7e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:29.994201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-be1d72c7-fab0-4545-b9d8-1f9012e098c1 None None] MFA Rules not processed for user `ff9b14ab630d4a928614c60e6aea667c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:29.995701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-be1d72c7-fab0-4545-b9d8-1f9012e098c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:30.064516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1164/2314] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:30.072680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1151/2315] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:30.077048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d72c7-fab0-4545-b9d8-1f9012e098c1 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:30.092058 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d72c7-fab0-4545-b9d8-1f9012e098c1 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:30.099408 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5586-fbef-412d-82f0-28c86c4e20fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:30.100187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5586-fbef-412d-82f0-28c86c4e20fc admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:30.100714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5586-fbef-412d-82f0-28c86c4e20fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:30.101079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5586-fbef-412d-82f0-28c86c4e20fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:30.101865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f5586-fbef-412d-82f0-28c86c4e20fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:30.104176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f5586-fbef-412d-82f0-28c86c4e20fc admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:30.138644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1165/2316] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:26:30 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:26:30.142770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477716ec-c78d-4a20-ad1d-c985840b7e6a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:30.154588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477716ec-c78d-4a20-ad1d-c985840b7e6a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:30.192773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:30.226437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c85e1c-7914-4a67-8ab6-5a94e1485bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:30.227153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c85e1c-7914-4a67-8ab6-5a94e1485bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:30.227497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c85e1c-7914-4a67-8ab6-5a94e1485bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:30.227835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c85e1c-7914-4a67-8ab6-5a94e1485bc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:30.228560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c85e1c-7914-4a67-8ab6-5a94e1485bc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:30.230621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c85e1c-7914-4a67-8ab6-5a94e1485bc7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:30.256385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1152/2317] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:26:30 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:26:30.260032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5586-fbef-412d-82f0-28c86c4e20fc admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:30.272383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5586-fbef-412d-82f0-28c86c4e20fc admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:30.278951 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1a052-ad5f-4995-9619-b39cc1ab2e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:30.279633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1a052-ad5f-4995-9619-b39cc1ab2e11 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:30.280138 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1a052-ad5f-4995-9619-b39cc1ab2e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:30.280413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1a052-ad5f-4995-9619-b39cc1ab2e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:30.281176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1a052-ad5f-4995-9619-b39cc1ab2e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:30.282840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1a052-ad5f-4995-9619-b39cc1ab2e11 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:30.296953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1166/2318] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:26:30.298144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c85e1c-7914-4a67-8ab6-5a94e1485bc7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:30.308984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c85e1c-7914-4a67-8ab6-5a94e1485bc7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:30.314827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224bd60-d545-43e5-bcf0-daacdd0cefb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:30.321114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224bd60-d545-43e5-bcf0-daacdd0cefb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:30.321625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224bd60-d545-43e5-bcf0-daacdd0cefb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:30.321999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224bd60-d545-43e5-bcf0-daacdd0cefb1 admin admin] PATH_INFO: `/v3/projects/b4002f10776f48c5a34aaa62ae41dd60/users/858e41026fd1418c8ba3c725d12de488/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:30.339347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224bd60-d545-43e5-bcf0-daacdd0cefb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4002f10776f48c5a34aaa62ae41dd60, user_id=858e41026fd1418c8ba3c725d12de488, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:30.342328 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e224bd60-d545-43e5-bcf0-daacdd0cefb1 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:30.361154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1153/2319] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:26:30 2022] PUT /identity/v3/projects/b4002f10776f48c5a34aaa62ae41dd60/users/858e41026fd1418c8ba3c725d12de488/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:26:30.365063 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1a052-ad5f-4995-9619-b39cc1ab2e11 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:30.378616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1a052-ad5f-4995-9619-b39cc1ab2e11 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:30.383988 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:30.447534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9803a-315e-4518-80b8-d771ed28c498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:30.448150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9803a-315e-4518-80b8-d771ed28c498 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:30.448428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9803a-315e-4518-80b8-d771ed28c498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:30.448698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9803a-315e-4518-80b8-d771ed28c498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:30.449341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9803a-315e-4518-80b8-d771ed28c498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:30.450981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9803a-315e-4518-80b8-d771ed28c498 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:30.462522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1167/2320] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:26:30.466811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224bd60-d545-43e5-bcf0-daacdd0cefb1 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:30.479125 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224bd60-d545-43e5-bcf0-daacdd0cefb1 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:30.484718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0896e-c924-4bd4-be8d-85d8c1a2e8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:30.485449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0896e-c924-4bd4-be8d-85d8c1a2e8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:30.485779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0896e-c924-4bd4-be8d-85d8c1a2e8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:30.486053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0896e-c924-4bd4-be8d-85d8c1a2e8c5 admin admin] PATH_INFO: `/v3/projects/b4002f10776f48c5a34aaa62ae41dd60/users/858e41026fd1418c8ba3c725d12de488/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:30.488527 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0896e-c924-4bd4-be8d-85d8c1a2e8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4002f10776f48c5a34aaa62ae41dd60, user_id=858e41026fd1418c8ba3c725d12de488, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:30.491388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0896e-c924-4bd4-be8d-85d8c1a2e8c5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:30.506751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1154/2321] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:26:30 2022] PUT /identity/v3/projects/b4002f10776f48c5a34aaa62ae41dd60/users/858e41026fd1418c8ba3c725d12de488/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:26:37.181389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8415596-5ace-42a4-beb6-8eeadc72e28a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:37.181905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8415596-5ace-42a4-beb6-8eeadc72e28a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:37.182037 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8415596-5ace-42a4-beb6-8eeadc72e28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:37.210859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c8415596-5ace-42a4-beb6-8eeadc72e28a None None] MFA Rules not processed for user `858e41026fd1418c8ba3c725d12de488`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:37.212075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c8415596-5ace-42a4-beb6-8eeadc72e28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:37.284357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1168/2322] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:37.296855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0896e-c924-4bd4-be8d-85d8c1a2e8c5 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:37.298618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-64b0896e-c924-4bd4-be8d-85d8c1a2e8c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:37.363009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0896e-c924-4bd4-be8d-85d8c1a2e8c5 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:37.367626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a5ef6-7de2-49ff-a3a2-5a4ff80dbb59 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bafea2367e4c7992354bb2c23e7d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:37.368452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a5ef6-7de2-49ff-a3a2-5a4ff80dbb59 service glance] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:37.368821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a5ef6-7de2-49ff-a3a2-5a4ff80dbb59 service glance] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:37.369212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a5ef6-7de2-49ff-a3a2-5a4ff80dbb59 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:37.428266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a5ef6-7de2-49ff-a3a2-5a4ff80dbb59 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:37.429839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a5ef6-7de2-49ff-a3a2-5a4ff80dbb59 service glance] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:37.439772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1155/2323] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:26:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:26:37.593925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f535-690f-4be2-a314-365685ad41bd None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:37.594266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f535-690f-4be2-a314-365685ad41bd None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:37.594607 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36f535-690f-4be2-a314-365685ad41bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:37.617216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7a36f535-690f-4be2-a314-365685ad41bd None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:37.618742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7a36f535-690f-4be2-a314-365685ad41bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:37.684667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1169/2324] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:26:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:37.780171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a5ef6-7de2-49ff-a3a2-5a4ff80dbb59 service glance] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:37.781671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c93a5ef6-7de2-49ff-a3a2-5a4ff80dbb59 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:37.835961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:37.869268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a5ef6-7de2-49ff-a3a2-5a4ff80dbb59 service glance] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:37.880437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c1f92-ea62-4fed-9bc4-668488b2684a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:37.881259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c1f92-ea62-4fed-9bc4-668488b2684a service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:37.881366 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c1f92-ea62-4fed-9bc4-668488b2684a service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:37.881637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c1f92-ea62-4fed-9bc4-668488b2684a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:37.891333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c1f92-ea62-4fed-9bc4-668488b2684a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:37.892967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c1f92-ea62-4fed-9bc4-668488b2684a service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:37.922048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1156/2325] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:26:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:26:42.071820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f535-690f-4be2-a314-365685ad41bd None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:42.073323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7a36f535-690f-4be2-a314-365685ad41bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:42.119254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:26:42.139579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36f535-690f-4be2-a314-365685ad41bd None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:42.144124 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051ec10-5adf-40a1-934a-a856f0c6ff5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:42.144915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ec10-5adf-40a1-934a-a856f0c6ff5a service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:42.145339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ec10-5adf-40a1-934a-a856f0c6ff5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:42.145798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051ec10-5adf-40a1-934a-a856f0c6ff5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:42.147558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a051ec10-5adf-40a1-934a-a856f0c6ff5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:42.363453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051ec10-5adf-40a1-934a-a856f0c6ff5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:42.365009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051ec10-5adf-40a1-934a-a856f0c6ff5a service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:42.377492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1170/2326] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:26:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:26:44.823914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c1f92-ea62-4fed-9bc4-668488b2684a service swift] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:44.835671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c1f92-ea62-4fed-9bc4-668488b2684a service swift] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:44.841204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95beb3c8-ed76-4ed2-b5e5-4d11b89efbb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:44.841966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95beb3c8-ed76-4ed2-b5e5-4d11b89efbb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:44.842347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95beb3c8-ed76-4ed2-b5e5-4d11b89efbb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:44.842775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95beb3c8-ed76-4ed2-b5e5-4d11b89efbb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:44.844722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-95beb3c8-ed76-4ed2-b5e5-4d11b89efbb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:44.863112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95beb3c8-ed76-4ed2-b5e5-4d11b89efbb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:44.864814 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95beb3c8-ed76-4ed2-b5e5-4d11b89efbb2 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:44.887591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1157/2327] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:26:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:26:48.472220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cbfb6-2646-417c-b3a1-495bdcffafaf None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:48.473075 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cbfb6-2646-417c-b3a1-495bdcffafaf None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:48.473557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cbfb6-2646-417c-b3a1-495bdcffafaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:48.494461 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-144cbfb6-2646-417c-b3a1-495bdcffafaf None None] MFA Rules not processed for user `5500db61c95f4245945784ae6b13f197`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:48.496264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-144cbfb6-2646-417c-b3a1-495bdcffafaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:48.509296 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1171/2328] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:49.864592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a676f80-9a3c-40cc-a20b-704d894af07b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:49.865065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a676f80-9a3c-40cc-a20b-704d894af07b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:49.865158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a676f80-9a3c-40cc-a20b-704d894af07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:49.885828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8a676f80-9a3c-40cc-a20b-704d894af07b None None] MFA Rules not processed for user `10d2803e109849fda2828f66f51b8a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:49.887007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8a676f80-9a3c-40cc-a20b-704d894af07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:49.942332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1158/2329] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:26:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:26:50.947897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca196b-713f-4516-bfe8-5ea6a5b5c12a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:50.948362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca196b-713f-4516-bfe8-5ea6a5b5c12a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:50.948612 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca196b-713f-4516-bfe8-5ea6a5b5c12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:50.973225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d8ca196b-713f-4516-bfe8-5ea6a5b5c12a None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:26:50.974701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d8ca196b-713f-4516-bfe8-5ea6a5b5c12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:51.000018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1172/2330] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:26:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:26:51.001104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a676f80-9a3c-40cc-a20b-704d894af07b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:51.018793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a676f80-9a3c-40cc-a20b-704d894af07b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:51.026787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5fd94-aa69-4701-82ae-ebd4d3a22f25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:51.027470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5fd94-aa69-4701-82ae-ebd4d3a22f25 service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:51.027757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5fd94-aa69-4701-82ae-ebd4d3a22f25 service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:51.028036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5fd94-aa69-4701-82ae-ebd4d3a22f25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:51.039474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5fd94-aa69-4701-82ae-ebd4d3a22f25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:51.041133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5fd94-aa69-4701-82ae-ebd4d3a22f25 service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:51.055143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1159/2331] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:26:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 05 20:26:54.570107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca196b-713f-4516-bfe8-5ea6a5b5c12a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:26:54.571397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d8ca196b-713f-4516-bfe8-5ea6a5b5c12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:54.587043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca196b-713f-4516-bfe8-5ea6a5b5c12a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:26:54.592297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceb7e3-5399-4c32-8808-06601c2f204c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:26:54.592880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceb7e3-5399-4c32-8808-06601c2f204c service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:26:54.593158 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceb7e3-5399-4c32-8808-06601c2f204c service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:26:54.593420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cceb7e3-5399-4c32-8808-06601c2f204c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:26:54.595092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4cceb7e3-5399-4c32-8808-06601c2f204c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:26:54.654417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cceb7e3-5399-4c32-8808-06601c2f204c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:26:54.656838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cceb7e3-5399-4c32-8808-06601c2f204c service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:26:54.665797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1173/2332] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:26:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:01.051355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=51 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:04.661599 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=72 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:05.046053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5fd94-aa69-4701-82ae-ebd4d3a22f25 service swift] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:05.058307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5fd94-aa69-4701-82ae-ebd4d3a22f25 service swift] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:05.064335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3a44d-8602-4254-8d0e-90f13090850e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:05.064930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3a44d-8602-4254-8d0e-90f13090850e service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:05.065211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3a44d-8602-4254-8d0e-90f13090850e service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:05.065485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3a44d-8602-4254-8d0e-90f13090850e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:05.067466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0bb3a44d-8602-4254-8d0e-90f13090850e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:05.125432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3a44d-8602-4254-8d0e-90f13090850e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:05.127128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3a44d-8602-4254-8d0e-90f13090850e service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:05.136652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1160/2333] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:13.208460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceb7e3-5399-4c32-8808-06601c2f204c service nova] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:13.222116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cceb7e3-5399-4c32-8808-06601c2f204c service nova] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:13.226007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867eb75-bda3-4554-ae20-2bd1f7c08bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:13.226879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867eb75-bda3-4554-ae20-2bd1f7c08bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:13.227474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867eb75-bda3-4554-ae20-2bd1f7c08bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:13.228262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867eb75-bda3-4554-ae20-2bd1f7c08bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:13.230789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e867eb75-bda3-4554-ae20-2bd1f7c08bb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:13.246450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867eb75-bda3-4554-ae20-2bd1f7c08bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:13.248002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867eb75-bda3-4554-ae20-2bd1f7c08bb0 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:13.265869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1174/2334] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:15.131965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=28 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:16.630534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3357-9475-4d43-95df-af2ed585293f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:16.631110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3357-9475-4d43-95df-af2ed585293f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:16.631110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e3357-9475-4d43-95df-af2ed585293f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:16.651135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-da5e3357-9475-4d43-95df-af2ed585293f None None] MFA Rules not processed for user `c28866ff5b2a4dd4ba05270b369f064e`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:16.652422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-da5e3357-9475-4d43-95df-af2ed585293f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:16.659237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1161/2335] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:16.966930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867eb75-bda3-4554-ae20-2bd1f7c08bb0 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:16.976112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867eb75-bda3-4554-ae20-2bd1f7c08bb0 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:16.979599 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:16.980186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:16.980457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:16.980718 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:16.982421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:16.991969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:16.993370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:17.001453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1175/2336] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:17.360153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e3357-9475-4d43-95df-af2ed585293f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:17.362058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-da5e3357-9475-4d43-95df-af2ed585293f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:17.472137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e3357-9475-4d43-95df-af2ed585293f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:17.478507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b06274-9ed2-48d8-ae60-ab70faa9b8e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:17.480239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b06274-9ed2-48d8-ae60-ab70faa9b8e6 service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:17.480699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b06274-9ed2-48d8-ae60-ab70faa9b8e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:17.481145 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b06274-9ed2-48d8-ae60-ab70faa9b8e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:17.484505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e5b06274-9ed2-48d8-ae60-ab70faa9b8e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:17.498718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b06274-9ed2-48d8-ae60-ab70faa9b8e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:17.501152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b06274-9ed2-48d8-ae60-ab70faa9b8e6 service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:17.513684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1162/2337] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:22.293140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:22.295434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:22.312860 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7baf1-f945-4d49-a95d-93f62abc1d23 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:22.319403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08609cbe-20be-4610-a1d0-a1cfe4ee1179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:22.320217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08609cbe-20be-4610-a1d0-a1cfe4ee1179 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:22.320499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08609cbe-20be-4610-a1d0-a1cfe4ee1179 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:22.320779 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08609cbe-20be-4610-a1d0-a1cfe4ee1179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:22.322546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-08609cbe-20be-4610-a1d0-a1cfe4ee1179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:22.335631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08609cbe-20be-4610-a1d0-a1cfe4ee1179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:22.337229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08609cbe-20be-4610-a1d0-a1cfe4ee1179 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:22.351536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1176/2338] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:24.303675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b06274-9ed2-48d8-ae60-ab70faa9b8e6 service placement] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:24.325041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b06274-9ed2-48d8-ae60-ab70faa9b8e6 service placement] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:24.333044 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633f564-f0d3-48dc-8fc8-a0509c8ed803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:24.334094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633f564-f0d3-48dc-8fc8-a0509c8ed803 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:24.334528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633f564-f0d3-48dc-8fc8-a0509c8ed803 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:24.334988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633f564-f0d3-48dc-8fc8-a0509c8ed803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:24.337420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8633f564-f0d3-48dc-8fc8-a0509c8ed803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:24.451334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633f564-f0d3-48dc-8fc8-a0509c8ed803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:24.454049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633f564-f0d3-48dc-8fc8-a0509c8ed803 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:24.485087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1163/2339] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:27:25.758168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08609cbe-20be-4610-a1d0-a1cfe4ee1179 service nova] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:25.776445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08609cbe-20be-4610-a1d0-a1cfe4ee1179 service nova] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:25.782821 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb6284-8316-4f38-b5fc-19900ffe5247 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:25.783977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6284-8316-4f38-b5fc-19900ffe5247 service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:25.784507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6284-8316-4f38-b5fc-19900ffe5247 service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:25.785099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6284-8316-4f38-b5fc-19900ffe5247 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:25.787290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c2eb6284-8316-4f38-b5fc-19900ffe5247 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:25.800879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb6284-8316-4f38-b5fc-19900ffe5247 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:25.802893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eb6284-8316-4f38-b5fc-19900ffe5247 service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:25.815307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1177/2340] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:30.932247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633f564-f0d3-48dc-8fc8-a0509c8ed803 service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:30.942329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633f564-f0d3-48dc-8fc8-a0509c8ed803 service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:30.945918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544b0d4-3607-45ca-98ef-413a5420dcd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:30.946512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544b0d4-3607-45ca-98ef-413a5420dcd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:30.946786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544b0d4-3607-45ca-98ef-413a5420dcd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:30.947054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544b0d4-3607-45ca-98ef-413a5420dcd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:30.956475 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544b0d4-3607-45ca-98ef-413a5420dcd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:30.958025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544b0d4-3607-45ca-98ef-413a5420dcd3 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:30.968136 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1164/2341] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:31.754919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb6284-8316-4f38-b5fc-19900ffe5247 service nova] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:31.764841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb6284-8316-4f38-b5fc-19900ffe5247 service nova] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:31.768470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e661f8e-7dea-48ac-a442-fb5a349bd6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:31.769045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e661f8e-7dea-48ac-a442-fb5a349bd6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:31.769318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e661f8e-7dea-48ac-a442-fb5a349bd6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:31.769614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e661f8e-7dea-48ac-a442-fb5a349bd6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:31.771508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1e661f8e-7dea-48ac-a442-fb5a349bd6d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:31.837853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e661f8e-7dea-48ac-a442-fb5a349bd6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:31.839387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e661f8e-7dea-48ac-a442-fb5a349bd6d1 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:31.848904 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1178/2342] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:35.699658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544b0d4-3607-45ca-98ef-413a5420dcd3 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:35.714193 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544b0d4-3607-45ca-98ef-413a5420dcd3 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:35.720630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffc43c-0a8c-43cc-b5de-0bb0de4be269 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:35.721467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffc43c-0a8c-43cc-b5de-0bb0de4be269 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:35.721903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffc43c-0a8c-43cc-b5de-0bb0de4be269 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:35.722273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffc43c-0a8c-43cc-b5de-0bb0de4be269 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:35.724225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f1ffc43c-0a8c-43cc-b5de-0bb0de4be269 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:35.743123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffc43c-0a8c-43cc-b5de-0bb0de4be269 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:35.745274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffc43c-0a8c-43cc-b5de-0bb0de4be269 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:35.758733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1165/2343] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:36.613629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2480-9c4a-4259-b35d-52ca156f8c64 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:36.614149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2480-9c4a-4259-b35d-52ca156f8c64 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:36.614299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2480-9c4a-4259-b35d-52ca156f8c64 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:36.615715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1179/2344] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:27:36.646950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b0957-41c7-4e47-9f95-0ee3e323d86c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:36.647321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b0957-41c7-4e47-9f95-0ee3e323d86c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:36.647667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b0957-41c7-4e47-9f95-0ee3e323d86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:36.669074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-584b0957-41c7-4e47-9f95-0ee3e323d86c None None] MFA Rules not processed for user `5181c4b31ac240469aff7a0aee35eb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:36.670922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-584b0957-41c7-4e47-9f95-0ee3e323d86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:36.681938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1166/2345] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:27:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:27:36.683197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2480-9c4a-4259-b35d-52ca156f8c64 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:36.692741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-07da2480-9c4a-4259-b35d-52ca156f8c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:36.696165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b0957-41c7-4e47-9f95-0ee3e323d86c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:36.702789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=55 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:36.713834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b0957-41c7-4e47-9f95-0ee3e323d86c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:36.714337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2480-9c4a-4259-b35d-52ca156f8c64 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:36.719980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:36.720264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebec6-13f5-4624-a542-baddd1813aa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:36.720876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebec6-13f5-4624-a542-baddd1813aa7 service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:36.721013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:36.721154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebec6-13f5-4624-a542-baddd1813aa7 service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:36.721253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:36.721425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebec6-13f5-4624-a542-baddd1813aa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:36.721544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] PATH_INFO: `/v3/users/1301a78c1a684c94acb9f26b56f39f21` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:36.722632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1301a78c1a684c94acb9f26b56f39f21)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:36.724561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:36.734101 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadebec6-13f5-4624-a542-baddd1813aa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:36.735881 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadebec6-13f5-4624-a542-baddd1813aa7 service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:36.748600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1167/2346] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:36.782657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1301a78c1a684c94acb9f26b56f39f21'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:36.783685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1301a78c1a684c94acb9f26b56f39f21'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:36.793340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1301a78c1a684c94acb9f26b56f39f21'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:36.794094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1301a78c1a684c94acb9f26b56f39f21'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:36.803243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e71ff7ad-8765-4692-bf82-ea94bb6285f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1301a78c1a684c94acb9f26b56f39f21'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:36.828835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1180/2347] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:27:36 2022] DELETE /identity/v3/users/1301a78c1a684c94acb9f26b56f39f21 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:37.009653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebec6-13f5-4624-a542-baddd1813aa7 service placement] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:37.022766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebec6-13f5-4624-a542-baddd1813aa7 service placement] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:37.055600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:37.081126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:37.081710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.082006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.082274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] PATH_INFO: `/v3/projects/07f868da50254a68bc692bca433d2699` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.083427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f868da50254a68bc692bca433d2699)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:37.085321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:37.130661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f868da50254a68bc692bca433d2699', 'request_id': 'req-e7499d05-57ed-4500-9521-d25e277d74f9'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:37.137571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f868da50254a68bc692bca433d2699', 'request_id': 'req-e7499d05-57ed-4500-9521-d25e277d74f9'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:37.143642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f868da50254a68bc692bca433d2699', 'request_id': 'req-e7499d05-57ed-4500-9521-d25e277d74f9'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:37.153875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] The token cache is being invalidate because project 07f868da50254a68bc692bca433d2699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:27:37.154155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:37.154783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7499d05-57ed-4500-9521-d25e277d74f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:37.156283 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1168/2348] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:27:37 2022] DELETE /identity/v3/projects/07f868da50254a68bc692bca433d2699 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:37.161272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ba1d7-92c6-467c-8e30-b5a863536709 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.161579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ba1d7-92c6-467c-8e30-b5a863536709 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.161849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ba1d7-92c6-467c-8e30-b5a863536709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.181683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-659ba1d7-92c6-467c-8e30-b5a863536709 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:37.182920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-659ba1d7-92c6-467c-8e30-b5a863536709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:37.241128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1181/2349] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:37.248839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29970c-ff8c-49ba-89b8-094d9cd69142 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.249122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29970c-ff8c-49ba-89b8-094d9cd69142 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.249394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29970c-ff8c-49ba-89b8-094d9cd69142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.268986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-1e29970c-ff8c-49ba-89b8-094d9cd69142 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:37.270204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1e29970c-ff8c-49ba-89b8-094d9cd69142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:37.277896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1169/2350] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:37.282382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ba1d7-92c6-467c-8e30-b5a863536709 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:37.291968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ba1d7-92c6-467c-8e30-b5a863536709 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:37.296498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a01aa-2914-4501-b7e4-4984185317ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:37.297083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a01aa-2914-4501-b7e4-4984185317ac admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.297358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a01aa-2914-4501-b7e4-4984185317ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.297673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a01aa-2914-4501-b7e4-4984185317ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.298329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a01aa-2914-4501-b7e4-4984185317ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:27:37.298791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a01aa-2914-4501-b7e4-4984185317ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:37.300382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a01aa-2914-4501-b7e4-4984185317ac admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:37.315537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1182/2351] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:27:37.320430 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29970c-ff8c-49ba-89b8-094d9cd69142 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:37.330385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29970c-ff8c-49ba-89b8-094d9cd69142 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:37.335071 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51210a43-d78f-4b9d-8727-e655491caff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:37.335670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51210a43-d78f-4b9d-8727-e655491caff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.335947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51210a43-d78f-4b9d-8727-e655491caff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.336216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51210a43-d78f-4b9d-8727-e655491caff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.336866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51210a43-d78f-4b9d-8727-e655491caff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:37.338775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51210a43-d78f-4b9d-8727-e655491caff0 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:37.365878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1170/2352] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:27:37 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:37.370105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a01aa-2914-4501-b7e4-4984185317ac admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:37.380218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a01aa-2914-4501-b7e4-4984185317ac admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:37.440715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b36d33-718c-4fd9-bec7-748588e0f6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:37.441323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b36d33-718c-4fd9-bec7-748588e0f6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.441631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b36d33-718c-4fd9-bec7-748588e0f6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.441904 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b36d33-718c-4fd9-bec7-748588e0f6f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.442797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b36d33-718c-4fd9-bec7-748588e0f6f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:37.444705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b36d33-718c-4fd9-bec7-748588e0f6f4 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:37.453644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:37.467726 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1183/2353] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:27:37 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:37.471790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51210a43-d78f-4b9d-8727-e655491caff0 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:37.482041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51210a43-d78f-4b9d-8727-e655491caff0 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:37.487202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95637-d530-4c02-b687-bf60518c6958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:37.487802 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95637-d530-4c02-b687-bf60518c6958 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.488079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95637-d530-4c02-b687-bf60518c6958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.488350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b95637-d530-4c02-b687-bf60518c6958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.488989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b95637-d530-4c02-b687-bf60518c6958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:37.490611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b95637-d530-4c02-b687-bf60518c6958 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:37.500598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1171/2354] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:37.504691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b36d33-718c-4fd9-bec7-748588e0f6f4 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:37.515324 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b36d33-718c-4fd9-bec7-748588e0f6f4 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:37.519976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43534d-6695-4d7b-bc6e-aad3407f4257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:37.520570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43534d-6695-4d7b-bc6e-aad3407f4257 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.520845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43534d-6695-4d7b-bc6e-aad3407f4257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.521129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43534d-6695-4d7b-bc6e-aad3407f4257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.521784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43534d-6695-4d7b-bc6e-aad3407f4257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:37.523373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43534d-6695-4d7b-bc6e-aad3407f4257 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:37.534265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1184/2355] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:37.538267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95637-d530-4c02-b687-bf60518c6958 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:37.549653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b95637-d530-4c02-b687-bf60518c6958 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:37.554467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7ca33-cf3b-4b09-8c21-db2a8567108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:37.555114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7ca33-cf3b-4b09-8c21-db2a8567108f admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.555391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7ca33-cf3b-4b09-8c21-db2a8567108f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.555663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7ca33-cf3b-4b09-8c21-db2a8567108f admin admin] PATH_INFO: `/v3/projects/a2077a9d747246eb9d17700fd494987e/users/d1e1393529ec4268add61bce90febb47/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.571697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7ca33-cf3b-4b09-8c21-db2a8567108f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2077a9d747246eb9d17700fd494987e, user_id=d1e1393529ec4268add61bce90febb47, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:37.574684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7ca33-cf3b-4b09-8c21-db2a8567108f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:37.587866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1172/2356] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:37 2022] PUT /identity/v3/projects/a2077a9d747246eb9d17700fd494987e/users/d1e1393529ec4268add61bce90febb47/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:37.601766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c25b3-6709-446c-8c7d-3a7c6f85a5fc None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.602054 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c25b3-6709-446c-8c7d-3a7c6f85a5fc None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.602330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c25b3-6709-446c-8c7d-3a7c6f85a5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.630169 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-964c25b3-6709-446c-8c7d-3a7c6f85a5fc None None] MFA Rules not processed for user `d1e1393529ec4268add61bce90febb47`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:37.631376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-964c25b3-6709-446c-8c7d-3a7c6f85a5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:37.694893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1185/2357] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:37.705916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7ca33-cf3b-4b09-8c21-db2a8567108f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:37.707580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9ba7ca33-cf3b-4b09-8c21-db2a8567108f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:37.766652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:37.783422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7ca33-cf3b-4b09-8c21-db2a8567108f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:37.789082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0de010-7109-4d0b-a46c-ddbbaefd27af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:37.789722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de010-7109-4d0b-a46c-ddbbaefd27af service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:37.790001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de010-7109-4d0b-a46c-ddbbaefd27af service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:37.790274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de010-7109-4d0b-a46c-ddbbaefd27af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:37.799071 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0de010-7109-4d0b-a46c-ddbbaefd27af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:37.800703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0de010-7109-4d0b-a46c-ddbbaefd27af service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:37.811547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1173/2358] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:40.538396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c25b3-6709-446c-8c7d-3a7c6f85a5fc None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:40.597040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c25b3-6709-446c-8c7d-3a7c6f85a5fc None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:40.601416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:40.602060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:40.602335 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:40.602603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] PATH_INFO: `/v3/users/d1e1393529ec4268add61bce90febb47` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:40.603713 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e1393529ec4268add61bce90febb47)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:40.605646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:40.652760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e1393529ec4268add61bce90febb47'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:40.653565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e1393529ec4268add61bce90febb47'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:40.660578 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e1393529ec4268add61bce90febb47'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:40.661202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e1393529ec4268add61bce90febb47'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:40.667871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e1393529ec4268add61bce90febb47'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:40.675168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:40.688099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1186/2359] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:27:40 2022] DELETE /identity/v3/users/d1e1393529ec4268add61bce90febb47 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:40.692960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42af171-2bea-4be1-a895-9ac460a7c934 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:40.693457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42af171-2bea-4be1-a895-9ac460a7c934 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:40.693592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42af171-2bea-4be1-a895-9ac460a7c934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:40.712877 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a42af171-2bea-4be1-a895-9ac460a7c934 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:40.714129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a42af171-2bea-4be1-a895-9ac460a7c934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:40.771418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1174/2360] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:40.780911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:40.782342 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:40.838074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d4181-6d53-4d1d-a934-1adcfa93adb6 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:40.841698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b6496-72d5-4e55-88aa-f092f1ee0427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:40.842331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b6496-72d5-4e55-88aa-f092f1ee0427 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:40.842608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b6496-72d5-4e55-88aa-f092f1ee0427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:40.842880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b6496-72d5-4e55-88aa-f092f1ee0427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:40.851586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b6496-72d5-4e55-88aa-f092f1ee0427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:40.852999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b6496-72d5-4e55-88aa-f092f1ee0427 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:40.863915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1187/2361] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:41.127156 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42af171-2bea-4be1-a895-9ac460a7c934 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.128572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a42af171-2bea-4be1-a895-9ac460a7c934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:41.141921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42af171-2bea-4be1-a895-9ac460a7c934 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.146514 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.147146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.147424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.148108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] PATH_INFO: `/v3/projects/a2077a9d747246eb9d17700fd494987e` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.148983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2077a9d747246eb9d17700fd494987e)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.150949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:41.206645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2077a9d747246eb9d17700fd494987e', 'request_id': 'req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:41.213944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2077a9d747246eb9d17700fd494987e', 'request_id': 'req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:41.220743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2077a9d747246eb9d17700fd494987e', 'request_id': 'req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:41.230856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] The token cache is being invalidate because project a2077a9d747246eb9d17700fd494987e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:27:41.231127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:41.231751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6f948b0e-4938-49ae-a3a1-8324bb5b1ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:41.233231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1175/2362] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:27:41 2022] DELETE /identity/v3/projects/a2077a9d747246eb9d17700fd494987e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:41.238066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3da026-7c31-4a61-b0ed-147f5dd40188 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.238344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3da026-7c31-4a61-b0ed-147f5dd40188 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.238610 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3da026-7c31-4a61-b0ed-147f5dd40188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.257869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-fb3da026-7c31-4a61-b0ed-147f5dd40188 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:41.259066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fb3da026-7c31-4a61-b0ed-147f5dd40188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:41.321641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1188/2363] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:41.330580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8bd4b-20c8-4f51-9ff5-cd0e306cfc86 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.330865 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8bd4b-20c8-4f51-9ff5-cd0e306cfc86 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.331161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8bd4b-20c8-4f51-9ff5-cd0e306cfc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.350771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-78a8bd4b-20c8-4f51-9ff5-cd0e306cfc86 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:41.352141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-78a8bd4b-20c8-4f51-9ff5-cd0e306cfc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:41.360400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1176/2364] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:41.364572 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3da026-7c31-4a61-b0ed-147f5dd40188 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.374996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3da026-7c31-4a61-b0ed-147f5dd40188 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.379427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d7606-ac8f-4e00-a707-9ded017eac9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.380018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d7606-ac8f-4e00-a707-9ded017eac9e admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.380290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d7606-ac8f-4e00-a707-9ded017eac9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.380552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d7606-ac8f-4e00-a707-9ded017eac9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.381187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d7606-ac8f-4e00-a707-9ded017eac9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:27:41.381479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d7606-ac8f-4e00-a707-9ded017eac9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.383066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8d7606-ac8f-4e00-a707-9ded017eac9e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:41.395495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1189/2365] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:27:41.399886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8bd4b-20c8-4f51-9ff5-cd0e306cfc86 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.409302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8bd4b-20c8-4f51-9ff5-cd0e306cfc86 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.413924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e2a9f-6d5b-451e-b77a-d8e85fe138bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.414520 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e2a9f-6d5b-451e-b77a-d8e85fe138bd admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.414854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e2a9f-6d5b-451e-b77a-d8e85fe138bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.415128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e2a9f-6d5b-451e-b77a-d8e85fe138bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.415755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e2a9f-6d5b-451e-b77a-d8e85fe138bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.417701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e2a9f-6d5b-451e-b77a-d8e85fe138bd admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:41.445792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1177/2366] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:27:41 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:41.449649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d7606-ac8f-4e00-a707-9ded017eac9e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.460074 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d7606-ac8f-4e00-a707-9ded017eac9e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.518394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30572c49-23cf-418e-aeea-29f0946a5c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.519000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30572c49-23cf-418e-aeea-29f0946a5c7e admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.519281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30572c49-23cf-418e-aeea-29f0946a5c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.519554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30572c49-23cf-418e-aeea-29f0946a5c7e admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.520189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30572c49-23cf-418e-aeea-29f0946a5c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.522140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30572c49-23cf-418e-aeea-29f0946a5c7e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:41.542653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1190/2367] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:27:41 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:41.547468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e2a9f-6d5b-451e-b77a-d8e85fe138bd admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.557791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e2a9f-6d5b-451e-b77a-d8e85fe138bd admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.562397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1edf5-4c14-4d77-81ae-2ea44e43eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.562987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1edf5-4c14-4d77-81ae-2ea44e43eec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.563265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1edf5-4c14-4d77-81ae-2ea44e43eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.563537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1edf5-4c14-4d77-81ae-2ea44e43eec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.564216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1edf5-4c14-4d77-81ae-2ea44e43eec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.565799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1edf5-4c14-4d77-81ae-2ea44e43eec1 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:41.575984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1178/2368] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:41.580201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30572c49-23cf-418e-aeea-29f0946a5c7e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.589899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30572c49-23cf-418e-aeea-29f0946a5c7e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.595030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc0772-a5f7-433a-a8cd-ee7d05c066a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.595610 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc0772-a5f7-433a-a8cd-ee7d05c066a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.595881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc0772-a5f7-433a-a8cd-ee7d05c066a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.596148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc0772-a5f7-433a-a8cd-ee7d05c066a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.596770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc0772-a5f7-433a-a8cd-ee7d05c066a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.598373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc0772-a5f7-433a-a8cd-ee7d05c066a2 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:41.600341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:41.610374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1191/2369] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:41.614170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1edf5-4c14-4d77-81ae-2ea44e43eec1 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.624648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1edf5-4c14-4d77-81ae-2ea44e43eec1 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.629237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb903483-f30f-46b6-b757-c079f6af982a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.629897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb903483-f30f-46b6-b757-c079f6af982a admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.630176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb903483-f30f-46b6-b757-c079f6af982a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.630450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb903483-f30f-46b6-b757-c079f6af982a admin admin] PATH_INFO: `/v3/projects/79151f27c2f34ef6a21bdfc9110c555b/users/c6b4bb6801ce4ea58de4eabee97b72c8/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.646706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb903483-f30f-46b6-b757-c079f6af982a admin admin] RBAC: Authorizing `identity:create_grant(project_id=79151f27c2f34ef6a21bdfc9110c555b, user_id=c6b4bb6801ce4ea58de4eabee97b72c8, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.650649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb903483-f30f-46b6-b757-c079f6af982a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:41.664344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1179/2370] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:41 2022] PUT /identity/v3/projects/79151f27c2f34ef6a21bdfc9110c555b/users/c6b4bb6801ce4ea58de4eabee97b72c8/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:27:41.676162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedd1df-0ea9-4f6f-975c-793929aa3ed2 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.676463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedd1df-0ea9-4f6f-975c-793929aa3ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.676721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedd1df-0ea9-4f6f-975c-793929aa3ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.704214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ffedd1df-0ea9-4f6f-975c-793929aa3ed2 None None] MFA Rules not processed for user `c6b4bb6801ce4ea58de4eabee97b72c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:41.705421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ffedd1df-0ea9-4f6f-975c-793929aa3ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:41.766700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1192/2371] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:41.771257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb903483-f30f-46b6-b757-c079f6af982a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.780644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:41.834182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb903483-f30f-46b6-b757-c079f6af982a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.838656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12133b47-86f8-406c-bcaf-208a10bf0b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.839248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12133b47-86f8-406c-bcaf-208a10bf0b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.839519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12133b47-86f8-406c-bcaf-208a10bf0b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.839786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12133b47-86f8-406c-bcaf-208a10bf0b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.840406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12133b47-86f8-406c-bcaf-208a10bf0b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.842324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12133b47-86f8-406c-bcaf-208a10bf0b80 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:41.869426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1180/2372] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:27:41 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:41.874147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedd1df-0ea9-4f6f-975c-793929aa3ed2 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.885048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedd1df-0ea9-4f6f-975c-793929aa3ed2 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.941383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6d69a-76d0-4892-8c15-fdf3711273ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.942013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6d69a-76d0-4892-8c15-fdf3711273ca admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.942545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6d69a-76d0-4892-8c15-fdf3711273ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.942813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6d69a-76d0-4892-8c15-fdf3711273ca admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.943467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6d69a-76d0-4892-8c15-fdf3711273ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.945368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6d69a-76d0-4892-8c15-fdf3711273ca admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:41.968137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1193/2373] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:27:41 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:41.972220 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12133b47-86f8-406c-bcaf-208a10bf0b80 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:41.982394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12133b47-86f8-406c-bcaf-208a10bf0b80 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:41.987604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fd130-f896-4f57-9d13-90dd448fe337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:41.988194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fd130-f896-4f57-9d13-90dd448fe337 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:41.988471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fd130-f896-4f57-9d13-90dd448fe337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:41.988744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fd130-f896-4f57-9d13-90dd448fe337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:41.989379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fd130-f896-4f57-9d13-90dd448fe337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:41.991103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fd130-f896-4f57-9d13-90dd448fe337 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:42.001365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1181/2374] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:42.005267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6d69a-76d0-4892-8c15-fdf3711273ca admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:42.015081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6d69a-76d0-4892-8c15-fdf3711273ca admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:42.019574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f0598-bf70-4718-b7c4-f6cc031b87a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:42.020205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f0598-bf70-4718-b7c4-f6cc031b87a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:42.020477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f0598-bf70-4718-b7c4-f6cc031b87a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:42.020743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f0598-bf70-4718-b7c4-f6cc031b87a0 admin admin] PATH_INFO: `/v3/projects/59ed378821b34065a5945de654030840/users/d69423df49c64e3f9fbd2eeda0ccbe73/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:42.035522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f0598-bf70-4718-b7c4-f6cc031b87a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ed378821b34065a5945de654030840, user_id=d69423df49c64e3f9fbd2eeda0ccbe73, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:42.038231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f0598-bf70-4718-b7c4-f6cc031b87a0 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:42.051185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1194/2375] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:42 2022] PUT /identity/v3/projects/59ed378821b34065a5945de654030840/users/d69423df49c64e3f9fbd2eeda0ccbe73/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:42.055078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fd130-f896-4f57-9d13-90dd448fe337 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:42.064742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fd130-f896-4f57-9d13-90dd448fe337 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:42.119898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e369085-b4df-4edf-a1e5-1e6aa5925f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:42.120498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e369085-b4df-4edf-a1e5-1e6aa5925f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:42.120776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e369085-b4df-4edf-a1e5-1e6aa5925f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:42.121047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e369085-b4df-4edf-a1e5-1e6aa5925f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:42.121711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e369085-b4df-4edf-a1e5-1e6aa5925f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:42.123838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e369085-b4df-4edf-a1e5-1e6aa5925f70 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:42.133716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1182/2376] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:42.137657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f0598-bf70-4718-b7c4-f6cc031b87a0 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:42.148359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f0598-bf70-4718-b7c4-f6cc031b87a0 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:42.152999 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5d565-8c0e-4dcb-9b50-2ea48c85df3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:42.153678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5d565-8c0e-4dcb-9b50-2ea48c85df3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:42.153957 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5d565-8c0e-4dcb-9b50-2ea48c85df3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:42.154227 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5d565-8c0e-4dcb-9b50-2ea48c85df3f admin admin] PATH_INFO: `/v3/projects/59ed378821b34065a5945de654030840/users/d69423df49c64e3f9fbd2eeda0ccbe73/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:42.156102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5d565-8c0e-4dcb-9b50-2ea48c85df3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ed378821b34065a5945de654030840, user_id=d69423df49c64e3f9fbd2eeda0ccbe73, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:42.158810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5d565-8c0e-4dcb-9b50-2ea48c85df3f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:42.168708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1195/2377] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:42 2022] PUT /identity/v3/projects/59ed378821b34065a5945de654030840/users/d69423df49c64e3f9fbd2eeda0ccbe73/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:42.178643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c82f0c-f9c6-4341-81cc-6aa5f8acab2f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:42.178928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c82f0c-f9c6-4341-81cc-6aa5f8acab2f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:42.179196 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c82f0c-f9c6-4341-81cc-6aa5f8acab2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:42.207470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-71c82f0c-f9c6-4341-81cc-6aa5f8acab2f None None] MFA Rules not processed for user `d69423df49c64e3f9fbd2eeda0ccbe73`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:42.208662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-71c82f0c-f9c6-4341-81cc-6aa5f8acab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:42.274107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1183/2378] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:42.284645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5d565-8c0e-4dcb-9b50-2ea48c85df3f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:42.286388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-70c5d565-8c0e-4dcb-9b50-2ea48c85df3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:42.330229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:42.345082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5d565-8c0e-4dcb-9b50-2ea48c85df3f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:42.348960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f60925-0911-45fc-bbcc-0ed7d9d67226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:42.349614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60925-0911-45fc-bbcc-0ed7d9d67226 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:42.349892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60925-0911-45fc-bbcc-0ed7d9d67226 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:42.350162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60925-0911-45fc-bbcc-0ed7d9d67226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:42.359493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f60925-0911-45fc-bbcc-0ed7d9d67226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:42.360920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f60925-0911-45fc-bbcc-0ed7d9d67226 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:42.370097 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1196/2379] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:43.700880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c82f0c-f9c6-4341-81cc-6aa5f8acab2f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:43.708788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:43.769750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c82f0c-f9c6-4341-81cc-6aa5f8acab2f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:43.774246 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:43.774875 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:43.775149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:43.775415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] PATH_INFO: `/v3/users/c6b4bb6801ce4ea58de4eabee97b72c8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:43.776521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6b4bb6801ce4ea58de4eabee97b72c8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:43.778456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:43.825580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6b4bb6801ce4ea58de4eabee97b72c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:43.826762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6b4bb6801ce4ea58de4eabee97b72c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:43.833825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6b4bb6801ce4ea58de4eabee97b72c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:43.841708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6b4bb6801ce4ea58de4eabee97b72c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:43.842928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6b4bb6801ce4ea58de4eabee97b72c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:43.867429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1184/2380] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:27:43 2022] DELETE /identity/v3/users/c6b4bb6801ce4ea58de4eabee97b72c8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:43.869266 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44f253-dc73-437d-abe8-79e9cb91fdf7 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:43.869770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44f253-dc73-437d-abe8-79e9cb91fdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:43.869865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44f253-dc73-437d-abe8-79e9cb91fdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:43.890533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ca44f253-dc73-437d-abe8-79e9cb91fdf7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:43.891732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ca44f253-dc73-437d-abe8-79e9cb91fdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:43.948457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1197/2381] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:43.958630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:43.960023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:44.017979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbe5fb-4819-49dd-a949-00dac4b4dbdd admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:44.021594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3523c-d46b-458f-84a0-ef10066e9868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:44.022227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3523c-d46b-458f-84a0-ef10066e9868 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:44.022502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3523c-d46b-458f-84a0-ef10066e9868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:44.022776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3523c-d46b-458f-84a0-ef10066e9868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:44.031598 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3523c-d46b-458f-84a0-ef10066e9868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:44.033007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3523c-d46b-458f-84a0-ef10066e9868 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:44.042356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1185/2382] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:27:44.311083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44f253-dc73-437d-abe8-79e9cb91fdf7 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:44.312474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ca44f253-dc73-437d-abe8-79e9cb91fdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:44.324448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44f253-dc73-437d-abe8-79e9cb91fdf7 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:44.329681 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:44.330303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:44.330595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:44.330857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] PATH_INFO: `/v3/projects/79151f27c2f34ef6a21bdfc9110c555b` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:44.331976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79151f27c2f34ef6a21bdfc9110c555b)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:44.333926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:44.380070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79151f27c2f34ef6a21bdfc9110c555b', 'request_id': 'req-9d35425a-8e60-4ec0-92ae-78ff881b1700'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.387260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79151f27c2f34ef6a21bdfc9110c555b', 'request_id': 'req-9d35425a-8e60-4ec0-92ae-78ff881b1700'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.394201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79151f27c2f34ef6a21bdfc9110c555b', 'request_id': 'req-9d35425a-8e60-4ec0-92ae-78ff881b1700'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.404123 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] The token cache is being invalidate because project 79151f27c2f34ef6a21bdfc9110c555b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:27:44.404405 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.405023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.406494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1198/2383] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:27:44 2022] DELETE /identity/v3/projects/79151f27c2f34ef6a21bdfc9110c555b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:44.411026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3523c-d46b-458f-84a0-ef10066e9868 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:44.420786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3523c-d46b-458f-84a0-ef10066e9868 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:44.440829 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:44.478788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:44.479407 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:44.479679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:44.479944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] PATH_INFO: `/v3/users/d69423df49c64e3f9fbd2eeda0ccbe73` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:44.481098 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d69423df49c64e3f9fbd2eeda0ccbe73)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:44.483094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:44.527869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd69423df49c64e3f9fbd2eeda0ccbe73'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.528519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd69423df49c64e3f9fbd2eeda0ccbe73'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.535832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd69423df49c64e3f9fbd2eeda0ccbe73'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.543011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd69423df49c64e3f9fbd2eeda0ccbe73'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.543653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-b6e27394-a1e6-4973-95ce-9ac9026e9d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd69423df49c64e3f9fbd2eeda0ccbe73'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.561163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1186/2384] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:27:44 2022] DELETE /identity/v3/users/d69423df49c64e3f9fbd2eeda0ccbe73 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:44.827184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:44.828618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:44.841071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35425a-8e60-4ec0-92ae-78ff881b1700 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:44.902946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:44.903571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:44.903844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:44.904112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] PATH_INFO: `/v3/projects/59ed378821b34065a5945de654030840` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:44.905328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ed378821b34065a5945de654030840)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:44.907474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:44.915739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:44.956153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ed378821b34065a5945de654030840', 'request_id': 'req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.967824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ed378821b34065a5945de654030840', 'request_id': 'req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.975490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ed378821b34065a5945de654030840', 'request_id': 'req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.987100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] The token cache is being invalidate because project 59ed378821b34065a5945de654030840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:27:44.987383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.988041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d441d7f7-65d2-40dc-95c7-b247c62e1fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:44.989730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1199/2385] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:27:44 2022] DELETE /identity/v3/projects/59ed378821b34065a5945de654030840 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:44.994815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbcfaa-a0aa-449b-a50f-fb3a39467e16 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:44.995338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbcfaa-a0aa-449b-a50f-fb3a39467e16 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:44.995428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbcfaa-a0aa-449b-a50f-fb3a39467e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.016791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-40fbcfaa-a0aa-449b-a50f-fb3a39467e16 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:45.018028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-40fbcfaa-a0aa-449b-a50f-fb3a39467e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:45.081043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1187/2386] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:45.088624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354817-16e5-48c9-8cc7-b2f8950a67f8 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.088905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354817-16e5-48c9-8cc7-b2f8950a67f8 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.089171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354817-16e5-48c9-8cc7-b2f8950a67f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.109218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-34354817-16e5-48c9-8cc7-b2f8950a67f8 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:45.110437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-34354817-16e5-48c9-8cc7-b2f8950a67f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:45.118116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1200/2387] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:45.122185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbcfaa-a0aa-449b-a50f-fb3a39467e16 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.131793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbcfaa-a0aa-449b-a50f-fb3a39467e16 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.136355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71548bf5-90e0-4591-83a8-bd3d5fd57167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.136948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71548bf5-90e0-4591-83a8-bd3d5fd57167 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.137223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71548bf5-90e0-4591-83a8-bd3d5fd57167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.137490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71548bf5-90e0-4591-83a8-bd3d5fd57167 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.138167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71548bf5-90e0-4591-83a8-bd3d5fd57167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:27:45.138469 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71548bf5-90e0-4591-83a8-bd3d5fd57167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.140045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71548bf5-90e0-4591-83a8-bd3d5fd57167 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.151332 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1188/2388] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:27:45.155958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34354817-16e5-48c9-8cc7-b2f8950a67f8 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.164803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34354817-16e5-48c9-8cc7-b2f8950a67f8 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.169353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a32bfb-077c-45ae-b26f-f3a0095b0aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.170491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a32bfb-077c-45ae-b26f-f3a0095b0aea admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.170787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a32bfb-077c-45ae-b26f-f3a0095b0aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.171090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a32bfb-077c-45ae-b26f-f3a0095b0aea admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.171721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a32bfb-077c-45ae-b26f-f3a0095b0aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.173618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a32bfb-077c-45ae-b26f-f3a0095b0aea admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.201000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1201/2389] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:27:45 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:45.205736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71548bf5-90e0-4591-83a8-bd3d5fd57167 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.217896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71548bf5-90e0-4591-83a8-bd3d5fd57167 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.247114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:45.275775 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29528026-5510-41be-aa61-4a92e4f905e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.276381 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29528026-5510-41be-aa61-4a92e4f905e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.276658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29528026-5510-41be-aa61-4a92e4f905e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.276931 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29528026-5510-41be-aa61-4a92e4f905e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.277578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29528026-5510-41be-aa61-4a92e4f905e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.280015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29528026-5510-41be-aa61-4a92e4f905e4 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.300911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1189/2390] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:27:45 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:45.305238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a32bfb-077c-45ae-b26f-f3a0095b0aea admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.316304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a32bfb-077c-45ae-b26f-f3a0095b0aea admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.321171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386c136-4ccb-459c-8464-5d24a51c6b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.321784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386c136-4ccb-459c-8464-5d24a51c6b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.322064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386c136-4ccb-459c-8464-5d24a51c6b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.322334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386c136-4ccb-459c-8464-5d24a51c6b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.322973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386c136-4ccb-459c-8464-5d24a51c6b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.324609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386c136-4ccb-459c-8464-5d24a51c6b9c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.335255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1202/2391] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:45.339034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29528026-5510-41be-aa61-4a92e4f905e4 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.348808 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29528026-5510-41be-aa61-4a92e4f905e4 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.353420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdafe9-8541-4cea-b4bd-9793299d42d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.354038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdafe9-8541-4cea-b4bd-9793299d42d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.354317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdafe9-8541-4cea-b4bd-9793299d42d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.354586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdafe9-8541-4cea-b4bd-9793299d42d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.355227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdafe9-8541-4cea-b4bd-9793299d42d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.356828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdafe9-8541-4cea-b4bd-9793299d42d6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.367832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1190/2392] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:27:45.371728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386c136-4ccb-459c-8464-5d24a51c6b9c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.381914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386c136-4ccb-459c-8464-5d24a51c6b9c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.386397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1182e1-0928-4cf8-b1f5-8f974c27f8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.387020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1182e1-0928-4cf8-b1f5-8f974c27f8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.387293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1182e1-0928-4cf8-b1f5-8f974c27f8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.387560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1182e1-0928-4cf8-b1f5-8f974c27f8a1 admin admin] PATH_INFO: `/v3/projects/5b5c41c11bc540878c3f640dfce47096/users/f914a5fe32dd496caef14b97fa3e212c/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.401899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1182e1-0928-4cf8-b1f5-8f974c27f8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b5c41c11bc540878c3f640dfce47096, user_id=f914a5fe32dd496caef14b97fa3e212c, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.404611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1182e1-0928-4cf8-b1f5-8f974c27f8a1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.416861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1203/2393] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:45 2022] PUT /identity/v3/projects/5b5c41c11bc540878c3f640dfce47096/users/f914a5fe32dd496caef14b97fa3e212c/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:45.428662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad87c44-e5b1-431c-bbe3-f0b296aeb28b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.428888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad87c44-e5b1-431c-bbe3-f0b296aeb28b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.429163 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad87c44-e5b1-431c-bbe3-f0b296aeb28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.456909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-fad87c44-e5b1-431c-bbe3-f0b296aeb28b None None] MFA Rules not processed for user `f914a5fe32dd496caef14b97fa3e212c`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:45.458147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fad87c44-e5b1-431c-bbe3-f0b296aeb28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:45.520813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1191/2394] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:45.525259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1182e1-0928-4cf8-b1f5-8f974c27f8a1 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.586302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1182e1-0928-4cf8-b1f5-8f974c27f8a1 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.591013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23817eca-5f96-4853-9330-c405e32569d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.591671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23817eca-5f96-4853-9330-c405e32569d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.592450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23817eca-5f96-4853-9330-c405e32569d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.592720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23817eca-5f96-4853-9330-c405e32569d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.593337 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23817eca-5f96-4853-9330-c405e32569d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.595261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23817eca-5f96-4853-9330-c405e32569d2 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.620533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1204/2395] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:27:45 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:27:45.624656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad87c44-e5b1-431c-bbe3-f0b296aeb28b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.635168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad87c44-e5b1-431c-bbe3-f0b296aeb28b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.690966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed18d4c-ca29-489b-8d62-b9dd48f93fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.691495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed18d4c-ca29-489b-8d62-b9dd48f93fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.691771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed18d4c-ca29-489b-8d62-b9dd48f93fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.692036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed18d4c-ca29-489b-8d62-b9dd48f93fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.692922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed18d4c-ca29-489b-8d62-b9dd48f93fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.694852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed18d4c-ca29-489b-8d62-b9dd48f93fd1 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.715143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1192/2396] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:27:45 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:45.719822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23817eca-5f96-4853-9330-c405e32569d2 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.729936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23817eca-5f96-4853-9330-c405e32569d2 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.734473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdeca0-2bdd-4c32-9725-b831e9a7e679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.735056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdeca0-2bdd-4c32-9725-b831e9a7e679 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.735330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdeca0-2bdd-4c32-9725-b831e9a7e679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.735598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdeca0-2bdd-4c32-9725-b831e9a7e679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.736231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cdeca0-2bdd-4c32-9725-b831e9a7e679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.737845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cdeca0-2bdd-4c32-9725-b831e9a7e679 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.748530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1205/2397] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:45.752574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed18d4c-ca29-489b-8d62-b9dd48f93fd1 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.762477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed18d4c-ca29-489b-8d62-b9dd48f93fd1 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.767642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d60cc-578d-4f89-b125-260b7afc336e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.768273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d60cc-578d-4f89-b125-260b7afc336e admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.768546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d60cc-578d-4f89-b125-260b7afc336e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.768816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d60cc-578d-4f89-b125-260b7afc336e admin admin] PATH_INFO: `/v3/projects/8e8865f81e18422280fb05b00ab7f55e/users/1d01612000ce4252bb3447371cab119c/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.783519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d60cc-578d-4f89-b125-260b7afc336e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8865f81e18422280fb05b00ab7f55e, user_id=1d01612000ce4252bb3447371cab119c, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.786161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d60cc-578d-4f89-b125-260b7afc336e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.798758 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1193/2398] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:45 2022] PUT /identity/v3/projects/8e8865f81e18422280fb05b00ab7f55e/users/1d01612000ce4252bb3447371cab119c/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:45.802818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdeca0-2bdd-4c32-9725-b831e9a7e679 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.813087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdeca0-2bdd-4c32-9725-b831e9a7e679 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.816558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:45.875130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3c6b1-61a0-4d29-8c00-32c1fe5b3797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.875753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3c6b1-61a0-4d29-8c00-32c1fe5b3797 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.876033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3c6b1-61a0-4d29-8c00-32c1fe5b3797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.876306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3c6b1-61a0-4d29-8c00-32c1fe5b3797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.876953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3c6b1-61a0-4d29-8c00-32c1fe5b3797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.879194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3c6b1-61a0-4d29-8c00-32c1fe5b3797 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.890010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1206/2399] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:45.894308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d60cc-578d-4f89-b125-260b7afc336e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:45.900886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:45.904787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d60cc-578d-4f89-b125-260b7afc336e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:45.910327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da55ecd-f2ba-4c80-ba3b-732f10eac797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:45.910961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da55ecd-f2ba-4c80-ba3b-732f10eac797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.911235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da55ecd-f2ba-4c80-ba3b-732f10eac797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.911498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da55ecd-f2ba-4c80-ba3b-732f10eac797 admin admin] PATH_INFO: `/v3/projects/8e8865f81e18422280fb05b00ab7f55e/users/1d01612000ce4252bb3447371cab119c/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.913348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da55ecd-f2ba-4c80-ba3b-732f10eac797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8865f81e18422280fb05b00ab7f55e, user_id=1d01612000ce4252bb3447371cab119c, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:45.915987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da55ecd-f2ba-4c80-ba3b-732f10eac797 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:45.926123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1194/2400] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:45 2022] PUT /identity/v3/projects/8e8865f81e18422280fb05b00ab7f55e/users/1d01612000ce4252bb3447371cab119c/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:45.936718 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283323bf-8b7c-496b-9f04-414f0982212c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:45.937195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283323bf-8b7c-496b-9f04-414f0982212c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:45.937308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283323bf-8b7c-496b-9f04-414f0982212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:45.964537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-283323bf-8b7c-496b-9f04-414f0982212c None None] MFA Rules not processed for user `1d01612000ce4252bb3447371cab119c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:45.965685 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-283323bf-8b7c-496b-9f04-414f0982212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:46.031032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1207/2401] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:46.041766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da55ecd-f2ba-4c80-ba3b-732f10eac797 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:46.043480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7da55ecd-f2ba-4c80-ba3b-732f10eac797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:46.100972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da55ecd-f2ba-4c80-ba3b-732f10eac797 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:46.104553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7db9e9-7f83-4362-b260-daa8b1f823a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:46.105155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db9e9-7f83-4362-b260-daa8b1f823a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:46.105432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db9e9-7f83-4362-b260-daa8b1f823a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:46.105728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7db9e9-7f83-4362-b260-daa8b1f823a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:46.115223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7db9e9-7f83-4362-b260-daa8b1f823a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:46.116642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7db9e9-7f83-4362-b260-daa8b1f823a1 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:46.126622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1195/2402] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:46.269143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283323bf-8b7c-496b-9f04-414f0982212c None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:46.329900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283323bf-8b7c-496b-9f04-414f0982212c None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:46.334430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:46.335049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:46.335328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:46.335602 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] PATH_INFO: `/v3/users/f914a5fe32dd496caef14b97fa3e212c` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:46.336702 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f914a5fe32dd496caef14b97fa3e212c)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:46.338646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:46.371099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:46.385412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f914a5fe32dd496caef14b97fa3e212c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.386092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f914a5fe32dd496caef14b97fa3e212c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.393318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f914a5fe32dd496caef14b97fa3e212c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.393959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f914a5fe32dd496caef14b97fa3e212c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.401148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f914a5fe32dd496caef14b97fa3e212c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.419814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1208/2403] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:27:46 2022] DELETE /identity/v3/users/f914a5fe32dd496caef14b97fa3e212c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:46.424326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c836a71-6403-4919-af43-f7a707990cd3 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:46.424552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c836a71-6403-4919-af43-f7a707990cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:46.424820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c836a71-6403-4919-af43-f7a707990cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:46.443316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-2c836a71-6403-4919-af43-f7a707990cd3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:46.444491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2c836a71-6403-4919-af43-f7a707990cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:46.501682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1196/2404] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:46.511174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:46.512583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:46.568787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8bd38-58c1-4ef8-a73a-483ed7e44ceb admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:46.572452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa83bbb-b298-4244-b0c7-95e4ad6e48f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:46.573048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa83bbb-b298-4244-b0c7-95e4ad6e48f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:46.573329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa83bbb-b298-4244-b0c7-95e4ad6e48f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:46.573642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa83bbb-b298-4244-b0c7-95e4ad6e48f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:46.583072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa83bbb-b298-4244-b0c7-95e4ad6e48f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:46.584495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa83bbb-b298-4244-b0c7-95e4ad6e48f2 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:46.593736 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1209/2405] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:46.856251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c836a71-6403-4919-af43-f7a707990cd3 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:46.857715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2c836a71-6403-4919-af43-f7a707990cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:46.875191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c836a71-6403-4919-af43-f7a707990cd3 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:46.880195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:46.880902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:46.881242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:46.881535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] PATH_INFO: `/v3/projects/5b5c41c11bc540878c3f640dfce47096` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:46.882666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b5c41c11bc540878c3f640dfce47096)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:46.884575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:46.932155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b5c41c11bc540878c3f640dfce47096', 'request_id': 'req-e5c2d043-ffcc-4348-a651-32c8e52d73c3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.939256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b5c41c11bc540878c3f640dfce47096', 'request_id': 'req-e5c2d043-ffcc-4348-a651-32c8e52d73c3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.945747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b5c41c11bc540878c3f640dfce47096', 'request_id': 'req-e5c2d043-ffcc-4348-a651-32c8e52d73c3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.956542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] The token cache is being invalidate because project 5b5c41c11bc540878c3f640dfce47096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:27:46.956828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.957461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:46.958966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1197/2406] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:27:46 2022] DELETE /identity/v3/projects/5b5c41c11bc540878c3f640dfce47096 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:46.962848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa83bbb-b298-4244-b0c7-95e4ad6e48f2 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:46.973129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa83bbb-b298-4244-b0c7-95e4ad6e48f2 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:47.028053 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:47.028606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:47.028886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:47.029161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] PATH_INFO: `/v3/users/1d01612000ce4252bb3447371cab119c` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:47.030307 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d01612000ce4252bb3447371cab119c)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:47.032807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:47.076925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d01612000ce4252bb3447371cab119c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.078104 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d01612000ce4252bb3447371cab119c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.082473 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:47.085543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d01612000ce4252bb3447371cab119c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.086396 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d01612000ce4252bb3447371cab119c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.095611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-92c7c493-994c-4741-89d4-93e50ae82c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d01612000ce4252bb3447371cab119c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.113048 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1210/2407] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:27:46 2022] DELETE /identity/v3/users/1d01612000ce4252bb3447371cab119c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:47.453285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:47.454915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:47.473170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2d043-ffcc-4348-a651-32c8e52d73c3 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:47.503901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:47.551262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:47.551829 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:47.552105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:47.552377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] PATH_INFO: `/v3/projects/8e8865f81e18422280fb05b00ab7f55e` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:47.553471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8865f81e18422280fb05b00ab7f55e)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:47.555404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:47.600855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8865f81e18422280fb05b00ab7f55e', 'request_id': 'req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.607670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8865f81e18422280fb05b00ab7f55e', 'request_id': 'req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.615909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8865f81e18422280fb05b00ab7f55e', 'request_id': 'req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.632725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] The token cache is being invalidate because project 8e8865f81e18422280fb05b00ab7f55e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:27:47.633016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.633706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-9c9d3c63-2fe1-4de1-a786-ba0269f84dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:27:47.635232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1198/2408] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:27:47 2022] DELETE /identity/v3/projects/8e8865f81e18422280fb05b00ab7f55e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:47.640060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3935ad1-f9dd-4630-8ecf-8bf932e81680 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:47.640365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3935ad1-f9dd-4630-8ecf-8bf932e81680 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:47.640631 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3935ad1-f9dd-4630-8ecf-8bf932e81680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:47.661667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b3935ad1-f9dd-4630-8ecf-8bf932e81680 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:47.662856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b3935ad1-f9dd-4630-8ecf-8bf932e81680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:47.719836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1211/2409] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:47.727841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff816eb-29dd-4f36-b602-44a4bab2c2af None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:47.728149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff816eb-29dd-4f36-b602-44a4bab2c2af None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:47.728427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff816eb-29dd-4f36-b602-44a4bab2c2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:47.748170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4ff816eb-29dd-4f36-b602-44a4bab2c2af None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:47.750042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4ff816eb-29dd-4f36-b602-44a4bab2c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:47.757848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1199/2410] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:47.762131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3935ad1-f9dd-4630-8ecf-8bf932e81680 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:47.771654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3935ad1-f9dd-4630-8ecf-8bf932e81680 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:47.776320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d41d8-6cac-49a8-9d27-ab2f4ec0eb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:47.776909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d41d8-6cac-49a8-9d27-ab2f4ec0eb31 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:47.777187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d41d8-6cac-49a8-9d27-ab2f4ec0eb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:47.777457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d41d8-6cac-49a8-9d27-ab2f4ec0eb31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:47.778131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d41d8-6cac-49a8-9d27-ab2f4ec0eb31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:27:47.778432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d41d8-6cac-49a8-9d27-ab2f4ec0eb31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:47.780022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d41d8-6cac-49a8-9d27-ab2f4ec0eb31 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:47.791328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1212/2411] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:27:47.795345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff816eb-29dd-4f36-b602-44a4bab2c2af None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:47.805399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff816eb-29dd-4f36-b602-44a4bab2c2af None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:47.810305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1790937-c8e9-41d2-aabf-936c2092a94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:47.810897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1790937-c8e9-41d2-aabf-936c2092a94d admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:47.811176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1790937-c8e9-41d2-aabf-936c2092a94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:47.811448 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1790937-c8e9-41d2-aabf-936c2092a94d admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:47.812615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1790937-c8e9-41d2-aabf-936c2092a94d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:47.814656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1790937-c8e9-41d2-aabf-936c2092a94d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:47.842935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1200/2412] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:27:47 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:47.846860 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d41d8-6cac-49a8-9d27-ab2f4ec0eb31 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:47.856798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d41d8-6cac-49a8-9d27-ab2f4ec0eb31 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:47.916438 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc8360-2bb2-402e-bd5c-2a3f1c079f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:47.917031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc8360-2bb2-402e-bd5c-2a3f1c079f91 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:47.917306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc8360-2bb2-402e-bd5c-2a3f1c079f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:47.917608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc8360-2bb2-402e-bd5c-2a3f1c079f91 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:47.918357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc8360-2bb2-402e-bd5c-2a3f1c079f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:47.920253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc8360-2bb2-402e-bd5c-2a3f1c079f91 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:47.941901 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1213/2413] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:27:47 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:47.946470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1790937-c8e9-41d2-aabf-936c2092a94d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:47.957177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1790937-c8e9-41d2-aabf-936c2092a94d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:47.961829 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29c863-9c7c-4f71-9e72-f1a59ca4ac73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:47.962424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29c863-9c7c-4f71-9e72-f1a59ca4ac73 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:47.962701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29c863-9c7c-4f71-9e72-f1a59ca4ac73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:47.962970 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29c863-9c7c-4f71-9e72-f1a59ca4ac73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:47.963609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29c863-9c7c-4f71-9e72-f1a59ca4ac73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:47.965185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29c863-9c7c-4f71-9e72-f1a59ca4ac73 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:47.975499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1201/2414] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:47.979129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc8360-2bb2-402e-bd5c-2a3f1c079f91 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:47.989418 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc8360-2bb2-402e-bd5c-2a3f1c079f91 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:47.994035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba10949-9b4e-452d-9db2-6f9f1e2b04bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:47.994611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba10949-9b4e-452d-9db2-6f9f1e2b04bb admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:47.994884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba10949-9b4e-452d-9db2-6f9f1e2b04bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:47.995148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba10949-9b4e-452d-9db2-6f9f1e2b04bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:47.995780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba10949-9b4e-452d-9db2-6f9f1e2b04bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:47.997421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba10949-9b4e-452d-9db2-6f9f1e2b04bb admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:48.008448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1214/2415] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:48.012652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29c863-9c7c-4f71-9e72-f1a59ca4ac73 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:48.022806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29c863-9c7c-4f71-9e72-f1a59ca4ac73 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:48.027434 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d23f3f-8c04-4369-bdff-368a08c88d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:48.028076 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d23f3f-8c04-4369-bdff-368a08c88d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:48.028355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d23f3f-8c04-4369-bdff-368a08c88d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:48.028629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d23f3f-8c04-4369-bdff-368a08c88d2d admin admin] PATH_INFO: `/v3/projects/ca1d8249d23347e8bf40ace9e93fb3c2/users/f1fb92c898304a908e8af0f28403a0b3/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:48.043405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d23f3f-8c04-4369-bdff-368a08c88d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1d8249d23347e8bf40ace9e93fb3c2, user_id=f1fb92c898304a908e8af0f28403a0b3, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:48.046680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d23f3f-8c04-4369-bdff-368a08c88d2d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:48.059134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1202/2416] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:48 2022] PUT /identity/v3/projects/ca1d8249d23347e8bf40ace9e93fb3c2/users/f1fb92c898304a908e8af0f28403a0b3/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:48.064195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a805272-efa2-4484-9d25-b18eea33e258 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:48.064530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a805272-efa2-4484-9d25-b18eea33e258 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:48.064800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a805272-efa2-4484-9d25-b18eea33e258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:48.083526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-2a805272-efa2-4484-9d25-b18eea33e258 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:48.084726 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2a805272-efa2-4484-9d25-b18eea33e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:48.115132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:48.146185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1215/2417] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:48.156266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d23f3f-8c04-4369-bdff-368a08c88d2d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:48.158057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-08d23f3f-8c04-4369-bdff-368a08c88d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:48.204902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:48.216549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d23f3f-8c04-4369-bdff-368a08c88d2d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:48.220880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef73bc-b7a1-4b13-bcaf-0a0d838a6175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:48.221477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef73bc-b7a1-4b13-bcaf-0a0d838a6175 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:48.221790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef73bc-b7a1-4b13-bcaf-0a0d838a6175 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:48.222057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef73bc-b7a1-4b13-bcaf-0a0d838a6175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:48.230973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef73bc-b7a1-4b13-bcaf-0a0d838a6175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:48.232378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef73bc-b7a1-4b13-bcaf-0a0d838a6175 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:48.242155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1203/2418] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:27:48.971298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8b792-cce8-4cba-b3ce-6c1c69de8808 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:48.971737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8b792-cce8-4cba-b3ce-6c1c69de8808 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:48.972116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8b792-cce8-4cba-b3ce-6c1c69de8808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:49.003737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-9ac8b792-cce8-4cba-b3ce-6c1c69de8808 None None] MFA Rules not processed for user `f1fb92c898304a908e8af0f28403a0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:49.005049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9ac8b792-cce8-4cba-b3ce-6c1c69de8808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:49.069938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1216/2419] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:27:49.074544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef73bc-b7a1-4b13-bcaf-0a0d838a6175 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:49.084616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef73bc-b7a1-4b13-bcaf-0a0d838a6175 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:49.089207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15d6b0-964a-425e-b296-ed7eb2d7993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:49.089920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15d6b0-964a-425e-b296-ed7eb2d7993f admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:49.090274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15d6b0-964a-425e-b296-ed7eb2d7993f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:49.090606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15d6b0-964a-425e-b296-ed7eb2d7993f admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:49.091295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15d6b0-964a-425e-b296-ed7eb2d7993f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:49.093770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15d6b0-964a-425e-b296-ed7eb2d7993f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:49.124713 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1204/2420] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:27:49 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:49.125853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8b792-cce8-4cba-b3ce-6c1c69de8808 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:49.136496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8b792-cce8-4cba-b3ce-6c1c69de8808 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:49.193375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9712f5-919d-4bae-b2dd-9a6130970606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:49.194022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9712f5-919d-4bae-b2dd-9a6130970606 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:49.194300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9712f5-919d-4bae-b2dd-9a6130970606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:49.194570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9712f5-919d-4bae-b2dd-9a6130970606 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:49.195204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9712f5-919d-4bae-b2dd-9a6130970606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:49.197154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9712f5-919d-4bae-b2dd-9a6130970606 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:49.219510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1217/2421] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:27:49 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:27:49.223742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15d6b0-964a-425e-b296-ed7eb2d7993f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:49.235256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15d6b0-964a-425e-b296-ed7eb2d7993f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:49.239840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4307995-0d10-472a-b915-b3c9a384352b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:49.240437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4307995-0d10-472a-b915-b3c9a384352b admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:49.240710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4307995-0d10-472a-b915-b3c9a384352b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:49.240976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4307995-0d10-472a-b915-b3c9a384352b admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:49.241635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4307995-0d10-472a-b915-b3c9a384352b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:49.243211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4307995-0d10-472a-b915-b3c9a384352b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:49.254576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1205/2422] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:49.258937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9712f5-919d-4bae-b2dd-9a6130970606 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:49.269004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9712f5-919d-4bae-b2dd-9a6130970606 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:49.273768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d363f1c-ace5-4545-a364-761887366942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:49.274443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d363f1c-ace5-4545-a364-761887366942 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:49.274723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d363f1c-ace5-4545-a364-761887366942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:49.274997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d363f1c-ace5-4545-a364-761887366942 admin admin] PATH_INFO: `/v3/projects/e9c5dee58033489daab82e182728c5b1/users/b173e0c7a8ce4411ad14446750b586b7/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:49.290073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d363f1c-ace5-4545-a364-761887366942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c5dee58033489daab82e182728c5b1, user_id=b173e0c7a8ce4411ad14446750b586b7, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:49.292739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d363f1c-ace5-4545-a364-761887366942 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:49.306521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1218/2423] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:49 2022] PUT /identity/v3/projects/e9c5dee58033489daab82e182728c5b1/users/b173e0c7a8ce4411ad14446750b586b7/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:27:49.310468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4307995-0d10-472a-b915-b3c9a384352b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:49.321583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4307995-0d10-472a-b915-b3c9a384352b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:49.380936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef91fba-b554-417c-8119-89c53fc2e21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:49.381602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef91fba-b554-417c-8119-89c53fc2e21f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:49.381885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef91fba-b554-417c-8119-89c53fc2e21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:49.382159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef91fba-b554-417c-8119-89c53fc2e21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:49.382823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef91fba-b554-417c-8119-89c53fc2e21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:49.384411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef91fba-b554-417c-8119-89c53fc2e21f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:49.399022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1206/2424] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:27:49.400166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d363f1c-ace5-4545-a364-761887366942 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:49.410513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d363f1c-ace5-4545-a364-761887366942 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:49.420092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d362f51-b01a-45bc-b8e4-00ad899cb9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:49.420756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d362f51-b01a-45bc-b8e4-00ad899cb9d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:49.421632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d362f51-b01a-45bc-b8e4-00ad899cb9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:49.421908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d362f51-b01a-45bc-b8e4-00ad899cb9d5 admin admin] PATH_INFO: `/v3/projects/e9c5dee58033489daab82e182728c5b1/users/b173e0c7a8ce4411ad14446750b586b7/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:49.423948 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d362f51-b01a-45bc-b8e4-00ad899cb9d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c5dee58033489daab82e182728c5b1, user_id=b173e0c7a8ce4411ad14446750b586b7, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:49.426643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d362f51-b01a-45bc-b8e4-00ad899cb9d5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:49.438096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1219/2425] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:27:49 2022] PUT /identity/v3/projects/e9c5dee58033489daab82e182728c5b1/users/b173e0c7a8ce4411ad14446750b586b7/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:27:50.157999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b64c7-c958-4cdd-942c-2308fcf69fe9 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:50.158477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b64c7-c958-4cdd-942c-2308fcf69fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:50.158592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b64c7-c958-4cdd-942c-2308fcf69fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:50.187793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-770b64c7-c958-4cdd-942c-2308fcf69fe9 None None] MFA Rules not processed for user `b173e0c7a8ce4411ad14446750b586b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:50.188990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-770b64c7-c958-4cdd-942c-2308fcf69fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:50.263105 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1207/2426] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:27:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:27:50.274392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d362f51-b01a-45bc-b8e4-00ad899cb9d5 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:50.276172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9d362f51-b01a-45bc-b8e4-00ad899cb9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:50.297826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:50.337030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d362f51-b01a-45bc-b8e4-00ad899cb9d5 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:50.340575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece771b-ed37-4356-908e-28b3a81879fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:50.341183 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece771b-ed37-4356-908e-28b3a81879fa service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:50.341463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece771b-ed37-4356-908e-28b3a81879fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:50.341765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece771b-ed37-4356-908e-28b3a81879fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:50.396708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece771b-ed37-4356-908e-28b3a81879fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:50.398235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece771b-ed37-4356-908e-28b3a81879fa service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:50.407322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1220/2427] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:27:51.622542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244393e-d412-40e6-9da1-5f945bad27e1 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:51.623030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244393e-d412-40e6-9da1-5f945bad27e1 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:51.623293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244393e-d412-40e6-9da1-5f945bad27e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:51.644063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f244393e-d412-40e6-9da1-5f945bad27e1 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:27:51.645330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f244393e-d412-40e6-9da1-5f945bad27e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:51.678859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:27:51.706309 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1208/2428] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:27:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:27:51.751484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece771b-ed37-4356-908e-28b3a81879fa service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:51.751484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-eece771b-ed37-4356-908e-28b3a81879fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:51.769344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244393e-d412-40e6-9da1-5f945bad27e1 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:27:51.770767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f244393e-d412-40e6-9da1-5f945bad27e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:27:51.781973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244393e-d412-40e6-9da1-5f945bad27e1 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:51.785618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893aba4-428c-4110-90a8-14ef3d576985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:51.786217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893aba4-428c-4110-90a8-14ef3d576985 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:51.786492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893aba4-428c-4110-90a8-14ef3d576985 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:51.786764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893aba4-428c-4110-90a8-14ef3d576985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:51.795635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893aba4-428c-4110-90a8-14ef3d576985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:51.797112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893aba4-428c-4110-90a8-14ef3d576985 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:51.806645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1209/2429] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:27:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:27:51.822216 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece771b-ed37-4356-908e-28b3a81879fa service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:27:51.827871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed3943-c2c6-485d-9eb0-d18cc18f457b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:27:51.828484 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed3943-c2c6-485d-9eb0-d18cc18f457b service swift] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:27:51.828763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed3943-c2c6-485d-9eb0-d18cc18f457b service swift] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:27:51.829035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed3943-c2c6-485d-9eb0-d18cc18f457b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:27:51.839568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed3943-c2c6-485d-9eb0-d18cc18f457b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:27:51.840832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed3943-c2c6-485d-9eb0-d18cc18f457b service swift] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:27:51.849658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1221/2430] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:27:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:28:01.802336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=15 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:01.846540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=72 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:05.651791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893aba4-428c-4110-90a8-14ef3d576985 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:05.722554 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893aba4-428c-4110-90a8-14ef3d576985 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:05.727109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:05.727729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:05.728006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:05.728272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] PATH_INFO: `/v3/users/f1fb92c898304a908e8af0f28403a0b3` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:05.729365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1fb92c898304a908e8af0f28403a0b3)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:05.731297 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:05.779213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1fb92c898304a908e8af0f28403a0b3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:05.779922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1fb92c898304a908e8af0f28403a0b3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:05.788285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1fb92c898304a908e8af0f28403a0b3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:05.795685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1fb92c898304a908e8af0f28403a0b3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:05.796377 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e9ef5763-4884-47cf-932d-f69d5cb75cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1fb92c898304a908e8af0f28403a0b3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:05.817005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1210/2431] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:28:05 2022] DELETE /identity/v3/users/f1fb92c898304a908e8af0f28403a0b3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:05.864850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd0a93-8f49-43c7-9eda-bdf6ae26c798 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:05.865395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd0a93-8f49-43c7-9eda-bdf6ae26c798 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:05.865568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd0a93-8f49-43c7-9eda-bdf6ae26c798 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:05.867104 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1222/2432] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:28:05.877263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959a111-aa73-4902-82a5-1bd5cd11daa3 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:05.877603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959a111-aa73-4902-82a5-1bd5cd11daa3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:05.877900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959a111-aa73-4902-82a5-1bd5cd11daa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:05.899343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-0959a111-aa73-4902-82a5-1bd5cd11daa3 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:05.900730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0959a111-aa73-4902-82a5-1bd5cd11daa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:05.984100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1211/2433] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:28:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:28:05.996156 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd0a93-8f49-43c7-9eda-bdf6ae26c798 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:05.998403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-51fd0a93-8f49-43c7-9eda-bdf6ae26c798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.029597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959a111-aa73-4902-82a5-1bd5cd11daa3 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:06.031130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0959a111-aa73-4902-82a5-1bd5cd11daa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.047739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:06.074200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd0a93-8f49-43c7-9eda-bdf6ae26c798 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:06.077661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f18380-afcc-4b57-89e4-65a3ce0dfcb6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bafea2367e4c7992354bb2c23e7d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:06.078813 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f18380-afcc-4b57-89e4-65a3ce0dfcb6 service glance] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.079092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f18380-afcc-4b57-89e4-65a3ce0dfcb6 service glance] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.079358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f18380-afcc-4b57-89e4-65a3ce0dfcb6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.089609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f18380-afcc-4b57-89e4-65a3ce0dfcb6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:06.091064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f18380-afcc-4b57-89e4-65a3ce0dfcb6 service glance] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:06.101878 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1223/2434] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:28:06.102974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959a111-aa73-4902-82a5-1bd5cd11daa3 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:06.107395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:06.108024 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.108300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.108572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] PATH_INFO: `/v3/projects/ca1d8249d23347e8bf40ace9e93fb3c2` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.109777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1d8249d23347e8bf40ace9e93fb3c2)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:06.111662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:06.160858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1d8249d23347e8bf40ace9e93fb3c2', 'request_id': 'req-76df9b8d-369d-4012-b3c1-336763215632'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.169133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1d8249d23347e8bf40ace9e93fb3c2', 'request_id': 'req-76df9b8d-369d-4012-b3c1-336763215632'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.172491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef9e66-95b6-4fb2-9b17-063f5d1db314 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.172764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef9e66-95b6-4fb2-9b17-063f5d1db314 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.173027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef9e66-95b6-4fb2-9b17-063f5d1db314 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.174379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1224/2435] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 20:28:06.176225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1d8249d23347e8bf40ace9e93fb3c2', 'request_id': 'req-76df9b8d-369d-4012-b3c1-336763215632'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.178661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f09698-86ea-4703-9392-28bbc526f933 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.178942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f09698-86ea-4703-9392-28bbc526f933 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.179206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f09698-86ea-4703-9392-28bbc526f933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.187129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] The token cache is being invalidate because project ca1d8249d23347e8bf40ace9e93fb3c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:28:06.187416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.188601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.190147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1212/2436] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:28:06 2022] DELETE /identity/v3/projects/ca1d8249d23347e8bf40ace9e93fb3c2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:28:06.194179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:06.195517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.202765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-93f09698-86ea-4703-9392-28bbc526f933 None None] MFA Rules not processed for user `8f5a88f263014f15a6f59cfdaa2bb6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:06.204548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-93f09698-86ea-4703-9392-28bbc526f933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.256415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1225/2437] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:28:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:28:06.261564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df9b8d-369d-4012-b3c1-336763215632 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:06.267213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:06.267839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.268114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.268442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] PATH_INFO: `/v3/users/b173e0c7a8ce4411ad14446750b586b7` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.269828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f09698-86ea-4703-9392-28bbc526f933 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:06.269967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b173e0c7a8ce4411ad14446750b586b7)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:06.271148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-93f09698-86ea-4703-9392-28bbc526f933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.271769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:06.281113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f09698-86ea-4703-9392-28bbc526f933 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:06.285389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d39d6-f0e3-42e6-9267-6a0e42a68bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:06.286012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d39d6-f0e3-42e6-9267-6a0e42a68bbd service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.286283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d39d6-f0e3-42e6-9267-6a0e42a68bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.286549 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d39d6-f0e3-42e6-9267-6a0e42a68bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.293531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d39d6-f0e3-42e6-9267-6a0e42a68bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:06.294944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d39d6-f0e3-42e6-9267-6a0e42a68bbd service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:06.303599 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1226/2438] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:28:06.318915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:06.321415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b173e0c7a8ce4411ad14446750b586b7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.322570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b173e0c7a8ce4411ad14446750b586b7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.331848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b173e0c7a8ce4411ad14446750b586b7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.339471 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b173e0c7a8ce4411ad14446750b586b7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.340116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d2e4958e-5a9c-45d7-9d1e-15256583360e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b173e0c7a8ce4411ad14446750b586b7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.361238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1213/2439] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:28:06 2022] DELETE /identity/v3/users/b173e0c7a8ce4411ad14446750b586b7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:28:06.390399 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247eb28-f1c3-40df-b3db-83003b3531e7 None None] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.391301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247eb28-f1c3-40df-b3db-83003b3531e7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.391657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247eb28-f1c3-40df-b3db-83003b3531e7 None None] PATH_INFO: `/` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.393061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1227/2440] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 20:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 20:28:06.398569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64dbd2-7183-4427-98ad-0ea544b0e036 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.398898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64dbd2-7183-4427-98ad-0ea544b0e036 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.399170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64dbd2-7183-4427-98ad-0ea544b0e036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.420103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-0b64dbd2-7183-4427-98ad-0ea544b0e036 None None] MFA Rules not processed for user `286f4e203e7a4a55bca0a58d061af2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:06.421238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0b64dbd2-7183-4427-98ad-0ea544b0e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.489137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1214/2441] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 20:28:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:06.499708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247eb28-f1c3-40df-b3db-83003b3531e7 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:06.501627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a247eb28-f1c3-40df-b3db-83003b3531e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.528420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64dbd2-7183-4427-98ad-0ea544b0e036 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:06.529886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0b64dbd2-7183-4427-98ad-0ea544b0e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.582998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247eb28-f1c3-40df-b3db-83003b3531e7 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:06.588760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d26b7-fa85-4959-a030-306ab24a20b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:06.589706 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d26b7-fa85-4959-a030-306ab24a20b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.590247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d26b7-fa85-4959-a030-306ab24a20b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.590535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d26b7-fa85-4959-a030-306ab24a20b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.603600 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d26b7-fa85-4959-a030-306ab24a20b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:06.605030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d26b7-fa85-4959-a030-306ab24a20b4 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:06.612376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64dbd2-7183-4427-98ad-0ea544b0e036 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:06.616750 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1228/2442] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:28:06.617241 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:06.617900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.618178 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.618444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] PATH_INFO: `/v3/projects/e9c5dee58033489daab82e182728c5b1` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.619547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c5dee58033489daab82e182728c5b1)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:06.621458 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:06.674882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c5dee58033489daab82e182728c5b1', 'request_id': 'req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.678599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:06.683944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c5dee58033489daab82e182728c5b1', 'request_id': 'req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.693500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c5dee58033489daab82e182728c5b1', 'request_id': 'req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.704994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] The token cache is being invalidate because project e9c5dee58033489daab82e182728c5b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:28:06.705306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.706005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f6f84b99-92b3-49b1-a93e-c6ced1447cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:06.708055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1215/2443] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:28:06 2022] DELETE /identity/v3/projects/e9c5dee58033489daab82e182728c5b1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:28:06.713500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1a480-7117-45a4-9e7c-4b9530ff2acb None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.714731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1a480-7117-45a4-9e7c-4b9530ff2acb None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.715204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1a480-7117-45a4-9e7c-4b9530ff2acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.737274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-f4b1a480-7117-45a4-9e7c-4b9530ff2acb None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:06.738870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f4b1a480-7117-45a4-9e7c-4b9530ff2acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.789573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:06.813092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1229/2444] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:28:06.822117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287330b-d1a6-4d72-ba54-f7ed24217998 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.822405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287330b-d1a6-4d72-ba54-f7ed24217998 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.822682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287330b-d1a6-4d72-ba54-f7ed24217998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.844862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-3287330b-d1a6-4d72-ba54-f7ed24217998 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:06.846111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3287330b-d1a6-4d72-ba54-f7ed24217998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:06.854040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1216/2445] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:28:06.858365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1a480-7117-45a4-9e7c-4b9530ff2acb None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:06.868993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1a480-7117-45a4-9e7c-4b9530ff2acb None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:06.879143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d13d2-31de-4d2c-916f-18c1fa931052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:06.879757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d13d2-31de-4d2c-916f-18c1fa931052 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.880170 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d13d2-31de-4d2c-916f-18c1fa931052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.880449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d13d2-31de-4d2c-916f-18c1fa931052 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.881125 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d13d2-31de-4d2c-916f-18c1fa931052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:28:06.881536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d13d2-31de-4d2c-916f-18c1fa931052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:06.883091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d13d2-31de-4d2c-916f-18c1fa931052 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:06.895809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1230/2446] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:28:06.900288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287330b-d1a6-4d72-ba54-f7ed24217998 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:06.910734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287330b-d1a6-4d72-ba54-f7ed24217998 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:06.915897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487215ed-fa7d-41db-b269-9e2311c8357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:06.916575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487215ed-fa7d-41db-b269-9e2311c8357c admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:06.916854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487215ed-fa7d-41db-b269-9e2311c8357c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:06.917187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487215ed-fa7d-41db-b269-9e2311c8357c admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:06.917786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487215ed-fa7d-41db-b269-9e2311c8357c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:06.919648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487215ed-fa7d-41db-b269-9e2311c8357c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:06.947218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1217/2447] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:28:06 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:28:06.951456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d13d2-31de-4d2c-916f-18c1fa931052 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:06.962181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d13d2-31de-4d2c-916f-18c1fa931052 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.021411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab2edc-7d51-42b1-a830-6e726b8412c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.022066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2edc-7d51-42b1-a830-6e726b8412c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.022346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2edc-7d51-42b1-a830-6e726b8412c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.022614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab2edc-7d51-42b1-a830-6e726b8412c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.023253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab2edc-7d51-42b1-a830-6e726b8412c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.025203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab2edc-7d51-42b1-a830-6e726b8412c2 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.049763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1231/2448] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:28:06 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:28:07.054014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487215ed-fa7d-41db-b269-9e2311c8357c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.065364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487215ed-fa7d-41db-b269-9e2311c8357c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.070499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251e533-9b8e-4554-a615-f2070c716f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.071115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251e533-9b8e-4554-a615-f2070c716f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.071532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251e533-9b8e-4554-a615-f2070c716f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.071806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251e533-9b8e-4554-a615-f2070c716f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.072453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251e533-9b8e-4554-a615-f2070c716f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.074099 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251e533-9b8e-4554-a615-f2070c716f08 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.085389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1218/2449] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:28:07.089433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab2edc-7d51-42b1-a830-6e726b8412c2 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.100056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab2edc-7d51-42b1-a830-6e726b8412c2 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.104898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eea4e-d9e4-48c1-9f8d-af5f3bf1ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.105533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eea4e-d9e4-48c1-9f8d-af5f3bf1ed5b admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.105818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eea4e-d9e4-48c1-9f8d-af5f3bf1ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.106089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eea4e-d9e4-48c1-9f8d-af5f3bf1ed5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.106732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50eea4e-d9e4-48c1-9f8d-af5f3bf1ed5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.108346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50eea4e-d9e4-48c1-9f8d-af5f3bf1ed5b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.120165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1232/2450] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:28:07.124341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251e533-9b8e-4554-a615-f2070c716f08 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.135782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251e533-9b8e-4554-a615-f2070c716f08 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.141131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a113d8c-1999-4b49-9e45-8a29204a5d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.141826 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a113d8c-1999-4b49-9e45-8a29204a5d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.142170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a113d8c-1999-4b49-9e45-8a29204a5d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.142447 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a113d8c-1999-4b49-9e45-8a29204a5d08 admin admin] PATH_INFO: `/v3/projects/4b41ba97c8c3440883248f16e082bafc/users/f2a5db82d6b14a688415ba3eb869a92c/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.160857 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a113d8c-1999-4b49-9e45-8a29204a5d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b41ba97c8c3440883248f16e082bafc, user_id=f2a5db82d6b14a688415ba3eb869a92c, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.163646 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a113d8c-1999-4b49-9e45-8a29204a5d08 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.178235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1219/2451] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:28:07 2022] PUT /identity/v3/projects/4b41ba97c8c3440883248f16e082bafc/users/f2a5db82d6b14a688415ba3eb869a92c/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:28:07.188819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434d5a1-7bdb-4108-ae4f-ed34b4f086b5 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.189105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434d5a1-7bdb-4108-ae4f-ed34b4f086b5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.189379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434d5a1-7bdb-4108-ae4f-ed34b4f086b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.220781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8434d5a1-7bdb-4108-ae4f-ed34b4f086b5 None None] MFA Rules not processed for user `f2a5db82d6b14a688415ba3eb869a92c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:07.222061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8434d5a1-7bdb-4108-ae4f-ed34b4f086b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:07.293000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1233/2452] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:07.297543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a113d8c-1999-4b49-9e45-8a29204a5d08 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.372164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a113d8c-1999-4b49-9e45-8a29204a5d08 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.377806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a8db-33ea-491f-b9b1-d55a26702e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.378587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a8db-33ea-491f-b9b1-d55a26702e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.378878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a8db-33ea-491f-b9b1-d55a26702e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.379154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a8db-33ea-491f-b9b1-d55a26702e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.379850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a8db-33ea-491f-b9b1-d55a26702e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.381763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a8db-33ea-491f-b9b1-d55a26702e8a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.414407 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1220/2453] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:28:07 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:28:07.418392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434d5a1-7bdb-4108-ae4f-ed34b4f086b5 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.429460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434d5a1-7bdb-4108-ae4f-ed34b4f086b5 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.468011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:07.492030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f028ff-3c02-49b0-8216-f630abd1f4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.492635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f028ff-3c02-49b0-8216-f630abd1f4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.492913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f028ff-3c02-49b0-8216-f630abd1f4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.493257 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f028ff-3c02-49b0-8216-f630abd1f4a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.493853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f028ff-3c02-49b0-8216-f630abd1f4a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.495767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f028ff-3c02-49b0-8216-f630abd1f4a8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.519535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1234/2454] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:28:07 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:28:07.523421 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a8db-33ea-491f-b9b1-d55a26702e8a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.534297 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a8db-33ea-491f-b9b1-d55a26702e8a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.539104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e28721-2f98-4b72-9af1-4a5a3d3be2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.539705 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e28721-2f98-4b72-9af1-4a5a3d3be2fc admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.539986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e28721-2f98-4b72-9af1-4a5a3d3be2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.540260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e28721-2f98-4b72-9af1-4a5a3d3be2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.540905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e28721-2f98-4b72-9af1-4a5a3d3be2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.542549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e28721-2f98-4b72-9af1-4a5a3d3be2fc admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.553704 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1221/2455] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:28:07.557300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f028ff-3c02-49b0-8216-f630abd1f4a8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.567649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f028ff-3c02-49b0-8216-f630abd1f4a8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.572308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33cd927-455c-483d-920e-88a70a62a41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.572875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cd927-455c-483d-920e-88a70a62a41a admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.573155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cd927-455c-483d-920e-88a70a62a41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.573427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cd927-455c-483d-920e-88a70a62a41a admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.574092 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33cd927-455c-483d-920e-88a70a62a41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.575679 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33cd927-455c-483d-920e-88a70a62a41a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.585656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1235/2456] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:28:07.589680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e28721-2f98-4b72-9af1-4a5a3d3be2fc admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.600002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e28721-2f98-4b72-9af1-4a5a3d3be2fc admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.604617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5dd35f-290e-4451-a7ee-e661f1d4992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.605256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dd35f-290e-4451-a7ee-e661f1d4992f admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.605562 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dd35f-290e-4451-a7ee-e661f1d4992f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.605840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dd35f-290e-4451-a7ee-e661f1d4992f admin admin] PATH_INFO: `/v3/projects/fa2970bd0cbb4883bde2450c4a12e985/users/658d3762dd88440ab9bb67aceee1addb/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.620521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5dd35f-290e-4451-a7ee-e661f1d4992f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2970bd0cbb4883bde2450c4a12e985, user_id=658d3762dd88440ab9bb67aceee1addb, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.623183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5dd35f-290e-4451-a7ee-e661f1d4992f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.638936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1222/2457] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:28:07 2022] PUT /identity/v3/projects/fa2970bd0cbb4883bde2450c4a12e985/users/658d3762dd88440ab9bb67aceee1addb/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:07.649922 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781ae31-3083-48e4-bf74-ffb1d7068dba None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.650203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781ae31-3083-48e4-bf74-ffb1d7068dba None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.650479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2781ae31-3083-48e4-bf74-ffb1d7068dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.678814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-2781ae31-3083-48e4-bf74-ffb1d7068dba None None] MFA Rules not processed for user `658d3762dd88440ab9bb67aceee1addb`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:07.680073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2781ae31-3083-48e4-bf74-ffb1d7068dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:07.742533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1236/2458] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:07.746982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5dd35f-290e-4451-a7ee-e661f1d4992f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.777837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:07.812118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5dd35f-290e-4451-a7ee-e661f1d4992f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.817815 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd712d-5a38-4a90-b6eb-7fe4aa63da26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.818429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd712d-5a38-4a90-b6eb-7fe4aa63da26 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.818896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd712d-5a38-4a90-b6eb-7fe4aa63da26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.819177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd712d-5a38-4a90-b6eb-7fe4aa63da26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.819805 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd712d-5a38-4a90-b6eb-7fe4aa63da26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.821741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbd712d-5a38-4a90-b6eb-7fe4aa63da26 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.853497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1223/2459] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:28:07 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:28:07.854796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781ae31-3083-48e4-bf74-ffb1d7068dba None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.866979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2781ae31-3083-48e4-bf74-ffb1d7068dba None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.932503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42919e-d16a-4f3b-b941-bf1d26492738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.933119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42919e-d16a-4f3b-b941-bf1d26492738 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.933369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42919e-d16a-4f3b-b941-bf1d26492738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.933670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42919e-d16a-4f3b-b941-bf1d26492738 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.934306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42919e-d16a-4f3b-b941-bf1d26492738 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.936202 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42919e-d16a-4f3b-b941-bf1d26492738 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.959314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1237/2460] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:28:07 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:28:07.963956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd712d-5a38-4a90-b6eb-7fe4aa63da26 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:07.974852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd712d-5a38-4a90-b6eb-7fe4aa63da26 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:07.979495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20d465-caad-4f51-bc13-e9335d5c6f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:07.980133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20d465-caad-4f51-bc13-e9335d5c6f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:07.980408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20d465-caad-4f51-bc13-e9335d5c6f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:07.980678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20d465-caad-4f51-bc13-e9335d5c6f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:07.981341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20d465-caad-4f51-bc13-e9335d5c6f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:07.982977 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20d465-caad-4f51-bc13-e9335d5c6f1f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:07.995637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1224/2461] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:28:07.999807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42919e-d16a-4f3b-b941-bf1d26492738 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:08.010699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42919e-d16a-4f3b-b941-bf1d26492738 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:08.015441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab9aa7-b37d-4536-93cd-09e1f2f4585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:08.016117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab9aa7-b37d-4536-93cd-09e1f2f4585d admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:08.016392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab9aa7-b37d-4536-93cd-09e1f2f4585d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:08.016660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab9aa7-b37d-4536-93cd-09e1f2f4585d admin admin] PATH_INFO: `/v3/projects/3ce06e19c061492c9a7d21e35e68da57/users/1dd7ba00190844afb602b20107756720/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:08.031384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab9aa7-b37d-4536-93cd-09e1f2f4585d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce06e19c061492c9a7d21e35e68da57, user_id=1dd7ba00190844afb602b20107756720, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:08.034085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab9aa7-b37d-4536-93cd-09e1f2f4585d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:08.060676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1238/2462] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:28:07 2022] PUT /identity/v3/projects/3ce06e19c061492c9a7d21e35e68da57/users/1dd7ba00190844afb602b20107756720/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:08.060896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20d465-caad-4f51-bc13-e9335d5c6f1f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:08.066906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20d465-caad-4f51-bc13-e9335d5c6f1f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:08.124858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee08e5-a977-4eb0-b202-d6a5ccd6bbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:08.126039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee08e5-a977-4eb0-b202-d6a5ccd6bbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:08.126321 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee08e5-a977-4eb0-b202-d6a5ccd6bbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:08.126600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee08e5-a977-4eb0-b202-d6a5ccd6bbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:08.127242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee08e5-a977-4eb0-b202-d6a5ccd6bbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:08.128832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee08e5-a977-4eb0-b202-d6a5ccd6bbd9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:08.138856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1225/2463] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:28:08.143649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab9aa7-b37d-4536-93cd-09e1f2f4585d admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:08.153624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab9aa7-b37d-4536-93cd-09e1f2f4585d admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:08.158132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65862-6fb0-4856-9ef8-980f01293571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:08.158769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65862-6fb0-4856-9ef8-980f01293571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:08.159042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65862-6fb0-4856-9ef8-980f01293571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:08.159308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65862-6fb0-4856-9ef8-980f01293571 admin admin] PATH_INFO: `/v3/projects/3ce06e19c061492c9a7d21e35e68da57/users/1dd7ba00190844afb602b20107756720/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:08.161177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da65862-6fb0-4856-9ef8-980f01293571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce06e19c061492c9a7d21e35e68da57, user_id=1dd7ba00190844afb602b20107756720, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:08.163820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da65862-6fb0-4856-9ef8-980f01293571 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:08.169220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:08.175187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1239/2464] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:28:08 2022] PUT /identity/v3/projects/3ce06e19c061492c9a7d21e35e68da57/users/1dd7ba00190844afb602b20107756720/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:08.240355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd3280-dc19-4437-a8dc-479d29612bc7 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:08.240763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd3280-dc19-4437-a8dc-479d29612bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:08.241111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd3280-dc19-4437-a8dc-479d29612bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:08.270889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-d5dd3280-dc19-4437-a8dc-479d29612bc7 None None] MFA Rules not processed for user `1dd7ba00190844afb602b20107756720`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:08.272091 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d5dd3280-dc19-4437-a8dc-479d29612bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:08.345876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1226/2465] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:08.357388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65862-6fb0-4856-9ef8-980f01293571 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:08.359260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9da65862-6fb0-4856-9ef8-980f01293571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:08.417416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65862-6fb0-4856-9ef8-980f01293571 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:08.421241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f30779-0e99-444b-bf93-8751570eaf78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:08.422481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f30779-0e99-444b-bf93-8751570eaf78 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:08.422763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f30779-0e99-444b-bf93-8751570eaf78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:08.423033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f30779-0e99-444b-bf93-8751570eaf78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:08.489229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f30779-0e99-444b-bf93-8751570eaf78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:08.490758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f30779-0e99-444b-bf93-8751570eaf78 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:08.500609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1240/2466] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:28:08.810896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9131f-c076-47c5-9178-58be9b317134 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:08.811354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9131f-c076-47c5-9178-58be9b317134 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:08.811612 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9131f-c076-47c5-9178-58be9b317134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:08.835301 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-72f9131f-c076-47c5-9178-58be9b317134 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:08.836599 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-72f9131f-c076-47c5-9178-58be9b317134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:08.860800 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:08.905828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1227/2467] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:28:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:28:08.916450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f30779-0e99-444b-bf93-8751570eaf78 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:08.918044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-77f30779-0e99-444b-bf93-8751570eaf78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:08.991547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f30779-0e99-444b-bf93-8751570eaf78 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:08.996332 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b721a-466b-42f8-897c-e804c4bd8406 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:08.996931 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b721a-466b-42f8-897c-e804c4bd8406 service swift] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:08.997207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b721a-466b-42f8-897c-e804c4bd8406 service swift] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:08.997481 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b721a-466b-42f8-897c-e804c4bd8406 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:09.005548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b721a-466b-42f8-897c-e804c4bd8406 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:09.007211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b721a-466b-42f8-897c-e804c4bd8406 service swift] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:09.015282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1241/2468] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:28:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:28:10.179418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9131f-c076-47c5-9178-58be9b317134 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:10.180965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-72f9131f-c076-47c5-9178-58be9b317134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:10.194707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9131f-c076-47c5-9178-58be9b317134 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:10.198501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d05b6-ddb2-41bd-9116-1b3934501a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:10.199652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d05b6-ddb2-41bd-9116-1b3934501a15 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:10.199933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d05b6-ddb2-41bd-9116-1b3934501a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:10.200204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d05b6-ddb2-41bd-9116-1b3934501a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:10.261939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d05b6-ddb2-41bd-9116-1b3934501a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:10.263470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d05b6-ddb2-41bd-9116-1b3934501a15 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:10.280251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1228/2469] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:28:10.940343 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b721a-466b-42f8-897c-e804c4bd8406 service swift] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:10.951013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b721a-466b-42f8-897c-e804c4bd8406 service swift] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:10.955928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca43ab-f7d7-478b-8d64-58c423c56291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:10.956806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca43ab-f7d7-478b-8d64-58c423c56291 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:10.957297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca43ab-f7d7-478b-8d64-58c423c56291 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:10.957977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca43ab-f7d7-478b-8d64-58c423c56291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:10.970313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca43ab-f7d7-478b-8d64-58c423c56291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:10.972529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca43ab-f7d7-478b-8d64-58c423c56291 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:10.984082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1242/2470] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:28:15.101456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d05b6-ddb2-41bd-9116-1b3934501a15 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:15.172133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d05b6-ddb2-41bd-9116-1b3934501a15 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:15.177095 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:15.177959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:15.178423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:15.178852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] PATH_INFO: `/v3/users/f2a5db82d6b14a688415ba3eb869a92c` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:15.180143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a5db82d6b14a688415ba3eb869a92c)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:15.182269 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:15.261774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a5db82d6b14a688415ba3eb869a92c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:15.262685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a5db82d6b14a688415ba3eb869a92c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:15.276010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a5db82d6b14a688415ba3eb869a92c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:15.286216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a5db82d6b14a688415ba3eb869a92c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:15.287029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a5db82d6b14a688415ba3eb869a92c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:15.296667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:15.316050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1229/2471] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:28:15 2022] DELETE /identity/v3/users/f2a5db82d6b14a688415ba3eb869a92c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:15.320635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07030c-5f83-4478-bf13-af9d8c38c127 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:15.321584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07030c-5f83-4478-bf13-af9d8c38c127 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:15.322021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07030c-5f83-4478-bf13-af9d8c38c127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:15.343434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5d07030c-5f83-4478-bf13-af9d8c38c127 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:15.345620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5d07030c-5f83-4478-bf13-af9d8c38c127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:15.386552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:15.439338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1243/2472] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:15.446579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:15.448159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:15.553785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4b521-b9d4-4d49-8294-7f6c673067a8 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:15.557937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799efde-8af4-4003-9e4d-3f6e8d78cf2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:15.558723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799efde-8af4-4003-9e4d-3f6e8d78cf2d service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:15.559003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799efde-8af4-4003-9e4d-3f6e8d78cf2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:15.559329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799efde-8af4-4003-9e4d-3f6e8d78cf2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:15.571382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799efde-8af4-4003-9e4d-3f6e8d78cf2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:15.572969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799efde-8af4-4003-9e4d-3f6e8d78cf2d service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:15.586181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1230/2473] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:28:15.957425 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07030c-5f83-4478-bf13-af9d8c38c127 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:15.958947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5d07030c-5f83-4478-bf13-af9d8c38c127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:15.972721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07030c-5f83-4478-bf13-af9d8c38c127 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:15.985286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:15.985973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:15.986250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:15.986625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] PATH_INFO: `/v3/projects/4b41ba97c8c3440883248f16e082bafc` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:15.988281 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b41ba97c8c3440883248f16e082bafc)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:15.996837 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:16.057907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b41ba97c8c3440883248f16e082bafc', 'request_id': 'req-811130f3-0037-4239-9297-eb1bdce748e6'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.067446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b41ba97c8c3440883248f16e082bafc', 'request_id': 'req-811130f3-0037-4239-9297-eb1bdce748e6'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.077260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b41ba97c8c3440883248f16e082bafc', 'request_id': 'req-811130f3-0037-4239-9297-eb1bdce748e6'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.089767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] The token cache is being invalidate because project 4b41ba97c8c3440883248f16e082bafc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:28:16.090041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.090762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.092610 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1244/2474] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:28:15 2022] DELETE /identity/v3/projects/4b41ba97c8c3440883248f16e082bafc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:16.097216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799efde-8af4-4003-9e4d-3f6e8d78cf2d service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:16.107437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799efde-8af4-4003-9e4d-3f6e8d78cf2d service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:16.167286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:16.167928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:16.168205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:16.168474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] PATH_INFO: `/v3/users/658d3762dd88440ab9bb67aceee1addb` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:16.169622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=658d3762dd88440ab9bb67aceee1addb)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:16.172113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:16.226740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '658d3762dd88440ab9bb67aceee1addb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.227398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '658d3762dd88440ab9bb67aceee1addb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.236517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '658d3762dd88440ab9bb67aceee1addb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.245637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '658d3762dd88440ab9bb67aceee1addb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.246331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '658d3762dd88440ab9bb67aceee1addb'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.264585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1231/2475] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:28:16 2022] DELETE /identity/v3/users/658d3762dd88440ab9bb67aceee1addb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:16.557373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:16.558887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:16.581444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811130f3-0037-4239-9297-eb1bdce748e6 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:16.643917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:16.644547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:16.644824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:16.645090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] PATH_INFO: `/v3/projects/fa2970bd0cbb4883bde2450c4a12e985` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:16.646530 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2970bd0cbb4883bde2450c4a12e985)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:16.648686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:16.730581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2970bd0cbb4883bde2450c4a12e985', 'request_id': 'req-23df17a8-835f-465a-b24d-83d1ef2c1c1a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.734796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:16.741907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2970bd0cbb4883bde2450c4a12e985', 'request_id': 'req-23df17a8-835f-465a-b24d-83d1ef2c1c1a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.762419 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2970bd0cbb4883bde2450c4a12e985', 'request_id': 'req-23df17a8-835f-465a-b24d-83d1ef2c1c1a'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.783103 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] The token cache is being invalidate because project fa2970bd0cbb4883bde2450c4a12e985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:28:16.783421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.784371 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.785656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1245/2476] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:28:16 2022] DELETE /identity/v3/projects/fa2970bd0cbb4883bde2450c4a12e985 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:16.789629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:16.791110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:16.804973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc1c4d-5983-4284-8b1b-b1895ab25941 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:16.810633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:16.872204 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:16.873272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:16.874038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:16.874370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] PATH_INFO: `/v3/users/1dd7ba00190844afb602b20107756720` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:16.876172 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd7ba00190844afb602b20107756720)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:16.879400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:16.950644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd7ba00190844afb602b20107756720'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.951299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd7ba00190844afb602b20107756720'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.959360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd7ba00190844afb602b20107756720'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.967536 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd7ba00190844afb602b20107756720'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.968256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-52f95ae7-97e9-4e9a-87b8-8e57db1229d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd7ba00190844afb602b20107756720'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:16.989665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1232/2477] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:28:16 2022] DELETE /identity/v3/users/1dd7ba00190844afb602b20107756720 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:17.368772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:17.370321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:17.383357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df17a8-835f-465a-b24d-83d1ef2c1c1a admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:17.443218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:17.443841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.444116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.444383 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] PATH_INFO: `/v3/projects/3ce06e19c061492c9a7d21e35e68da57` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.445486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce06e19c061492c9a7d21e35e68da57)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:17.447419 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:17.492958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce06e19c061492c9a7d21e35e68da57', 'request_id': 'req-47736118-87f2-49fc-96e4-c3a722af7971'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:17.499368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce06e19c061492c9a7d21e35e68da57', 'request_id': 'req-47736118-87f2-49fc-96e4-c3a722af7971'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:17.505915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce06e19c061492c9a7d21e35e68da57', 'request_id': 'req-47736118-87f2-49fc-96e4-c3a722af7971'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:17.518780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] The token cache is being invalidate because project 3ce06e19c061492c9a7d21e35e68da57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:28:17.519045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:17.519640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-47736118-87f2-49fc-96e4-c3a722af7971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:17.521121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1246/2478] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:28:17 2022] DELETE /identity/v3/projects/3ce06e19c061492c9a7d21e35e68da57 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:17.525993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a3306-158a-4815-8b70-30e90723d32b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.526497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a3306-158a-4815-8b70-30e90723d32b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.526611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a3306-158a-4815-8b70-30e90723d32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.545739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-165a3306-158a-4815-8b70-30e90723d32b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:17.547486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-165a3306-158a-4815-8b70-30e90723d32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:17.605829 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1233/2479] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:17.614115 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22aa75-fe39-4bee-a41a-0447080b632c None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.614627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22aa75-fe39-4bee-a41a-0447080b632c None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.614735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22aa75-fe39-4bee-a41a-0447080b632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.634875 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-af22aa75-fe39-4bee-a41a-0447080b632c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:17.636145 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-af22aa75-fe39-4bee-a41a-0447080b632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:17.645747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1247/2480] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:17.648452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a3306-158a-4815-8b70-30e90723d32b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:17.658112 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a3306-158a-4815-8b70-30e90723d32b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:17.662647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd80668-dd5c-4aea-a463-fdb536a4eeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:17.663250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd80668-dd5c-4aea-a463-fdb536a4eeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.663528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd80668-dd5c-4aea-a463-fdb536a4eeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.663798 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd80668-dd5c-4aea-a463-fdb536a4eeb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.664435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd80668-dd5c-4aea-a463-fdb536a4eeb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:28:17.664759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd80668-dd5c-4aea-a463-fdb536a4eeb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:17.666396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd80668-dd5c-4aea-a463-fdb536a4eeb3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:17.678403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1234/2481] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:28:17.682733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22aa75-fe39-4bee-a41a-0447080b632c None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:17.692601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22aa75-fe39-4bee-a41a-0447080b632c None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:17.697161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39d2c2-aeee-4f24-ad28-c2e31fb55a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:17.697789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39d2c2-aeee-4f24-ad28-c2e31fb55a22 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.698065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39d2c2-aeee-4f24-ad28-c2e31fb55a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.698332 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39d2c2-aeee-4f24-ad28-c2e31fb55a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.698954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39d2c2-aeee-4f24-ad28-c2e31fb55a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:17.700847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39d2c2-aeee-4f24-ad28-c2e31fb55a22 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:17.726870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1248/2482] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:28:17 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:28:17.731192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd80668-dd5c-4aea-a463-fdb536a4eeb3 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:17.741748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd80668-dd5c-4aea-a463-fdb536a4eeb3 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:17.753840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:17.800853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1628691-f45e-4823-9127-77d5af004833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:17.801381 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628691-f45e-4823-9127-77d5af004833 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.801687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628691-f45e-4823-9127-77d5af004833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.801961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628691-f45e-4823-9127-77d5af004833 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.802587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1628691-f45e-4823-9127-77d5af004833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:17.804529 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1628691-f45e-4823-9127-77d5af004833 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:17.825145 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1235/2483] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:28:17 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:28:17.829947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39d2c2-aeee-4f24-ad28-c2e31fb55a22 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:17.840329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39d2c2-aeee-4f24-ad28-c2e31fb55a22 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:17.845505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0eb6e1-34eb-4420-83c2-bb5a91fda469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:17.846126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0eb6e1-34eb-4420-83c2-bb5a91fda469 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.846407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0eb6e1-34eb-4420-83c2-bb5a91fda469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.846678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0eb6e1-34eb-4420-83c2-bb5a91fda469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.847315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0eb6e1-34eb-4420-83c2-bb5a91fda469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:17.848908 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0eb6e1-34eb-4420-83c2-bb5a91fda469 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:17.858640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1249/2484] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:28:17.862538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1628691-f45e-4823-9127-77d5af004833 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:17.872946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1628691-f45e-4823-9127-77d5af004833 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:17.878190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9fcfd-cf28-4334-bffa-62dc45075a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:17.878949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9fcfd-cf28-4334-bffa-62dc45075a36 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.879238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9fcfd-cf28-4334-bffa-62dc45075a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.879525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9fcfd-cf28-4334-bffa-62dc45075a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.880153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9fcfd-cf28-4334-bffa-62dc45075a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:17.881914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9fcfd-cf28-4334-bffa-62dc45075a36 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:17.894749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1236/2485] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:28:17.899183 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0eb6e1-34eb-4420-83c2-bb5a91fda469 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:17.909534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0eb6e1-34eb-4420-83c2-bb5a91fda469 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:17.914133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6108ab-4bc9-4952-9cde-817782b96ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:17.914777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6108ab-4bc9-4952-9cde-817782b96ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.915056 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6108ab-4bc9-4952-9cde-817782b96ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.915404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6108ab-4bc9-4952-9cde-817782b96ad8 admin admin] PATH_INFO: `/v3/projects/e17091ff17eb41388de7f3c2c97c2306/users/f7927b0047104ef08ef44b2c86aa4609/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.933762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6108ab-4bc9-4952-9cde-817782b96ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17091ff17eb41388de7f3c2c97c2306, user_id=f7927b0047104ef08ef44b2c86aa4609, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:17.937743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6108ab-4bc9-4952-9cde-817782b96ad8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:17.954790 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1250/2486] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:28:17 2022] PUT /identity/v3/projects/e17091ff17eb41388de7f3c2c97c2306/users/f7927b0047104ef08ef44b2c86aa4609/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:17.966164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955bc10-b9cf-4b31-87c2-1dcb69d8e597 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:17.966495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955bc10-b9cf-4b31-87c2-1dcb69d8e597 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:17.966813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955bc10-b9cf-4b31-87c2-1dcb69d8e597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:17.994614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-3955bc10-b9cf-4b31-87c2-1dcb69d8e597 None None] MFA Rules not processed for user `f7927b0047104ef08ef44b2c86aa4609`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:17.995848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3955bc10-b9cf-4b31-87c2-1dcb69d8e597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:18.061888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1237/2487] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:18.073634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6108ab-4bc9-4952-9cde-817782b96ad8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:18.075321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8a6108ab-4bc9-4952-9cde-817782b96ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:18.086758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:18.143905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6108ab-4bc9-4952-9cde-817782b96ad8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:18.147575 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6f2b4-d502-419a-b85e-ec9b991c963a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:18.148181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6f2b4-d502-419a-b85e-ec9b991c963a service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:18.148460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6f2b4-d502-419a-b85e-ec9b991c963a service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:18.148733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6f2b4-d502-419a-b85e-ec9b991c963a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:18.157811 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6f2b4-d502-419a-b85e-ec9b991c963a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:18.159232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6f2b4-d502-419a-b85e-ec9b991c963a service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:18.167975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1251/2488] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:28:25.006486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84107eb2-815d-40b3-b605-94639850eb64 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:25.007015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84107eb2-815d-40b3-b605-94639850eb64 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:25.007015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84107eb2-815d-40b3-b605-94639850eb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:25.030072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-84107eb2-815d-40b3-b605-94639850eb64 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:25.031872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-84107eb2-815d-40b3-b605-94639850eb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:25.090054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1238/2489] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 20:28:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:25.103456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6f2b4-d502-419a-b85e-ec9b991c963a service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:25.105302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f5b6f2b4-d502-419a-b85e-ec9b991c963a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:25.174891 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6f2b4-d502-419a-b85e-ec9b991c963a service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:25.179938 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed621764-be0c-448c-9bcf-26813cda7241 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:25.180532 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed621764-be0c-448c-9bcf-26813cda7241 service swift] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:25.180810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed621764-be0c-448c-9bcf-26813cda7241 service swift] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:25.181078 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed621764-be0c-448c-9bcf-26813cda7241 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:25.188880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed621764-be0c-448c-9bcf-26813cda7241 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:25.190545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed621764-be0c-448c-9bcf-26813cda7241 service swift] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:25.198643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1252/2490] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:28:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:28:35.083239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:35.195447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=54 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:37.544557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84107eb2-815d-40b3-b605-94639850eb64 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:37.546088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-84107eb2-815d-40b3-b605-94639850eb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:37.618379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84107eb2-815d-40b3-b605-94639850eb64 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:37.624004 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178770a2-fd99-4d07-a73d-e9b02d163294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:37.624632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178770a2-fd99-4d07-a73d-e9b02d163294 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:37.624911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178770a2-fd99-4d07-a73d-e9b02d163294 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:37.625191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178770a2-fd99-4d07-a73d-e9b02d163294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:37.626900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-178770a2-fd99-4d07-a73d-e9b02d163294 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:37.687732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178770a2-fd99-4d07-a73d-e9b02d163294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:37.689331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178770a2-fd99-4d07-a73d-e9b02d163294 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:37.698548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1239/2491] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:28:44.585764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f481de-f1f0-4a81-97d8-ae047beb7f05 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:44.586303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f481de-f1f0-4a81-97d8-ae047beb7f05 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:44.586476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f481de-f1f0-4a81-97d8-ae047beb7f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:44.605196 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-23f481de-f1f0-4a81-97d8-ae047beb7f05 None None] MFA Rules not processed for user `10d2803e109849fda2828f66f51b8a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:44.606449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-23f481de-f1f0-4a81-97d8-ae047beb7f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:44.613998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1253/2492] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:47.694369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=56 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:54.600027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:56.716968 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178770a2-fd99-4d07-a73d-e9b02d163294 service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:56.783725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178770a2-fd99-4d07-a73d-e9b02d163294 service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:56.788253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:56.788874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:56.789154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:56.789424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] PATH_INFO: `/v3/users/f7927b0047104ef08ef44b2c86aa4609` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:56.790577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7927b0047104ef08ef44b2c86aa4609)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:56.792487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:56.838939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7927b0047104ef08ef44b2c86aa4609'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:56.839584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7927b0047104ef08ef44b2c86aa4609'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:56.846479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7927b0047104ef08ef44b2c86aa4609'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:56.853614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7927b0047104ef08ef44b2c86aa4609'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:56.854290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7927b0047104ef08ef44b2c86aa4609'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:56.875123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1240/2493] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:28:56 2022] DELETE /identity/v3/users/f7927b0047104ef08ef44b2c86aa4609 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:56.880062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b9114-ae28-4a4a-9e93-a1452218dde5 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:56.880478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b9114-ae28-4a4a-9e93-a1452218dde5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:56.880606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b9114-ae28-4a4a-9e93-a1452218dde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:56.900796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-641b9114-ae28-4a4a-9e93-a1452218dde5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:56.902036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-641b9114-ae28-4a4a-9e93-a1452218dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:56.962265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1254/2494] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:56.974053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:56.975622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:57.056450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d2fd4-116d-43db-b728-0a3d6b354b18 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:57.060670 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d2051-b04f-4a63-aebd-deb5d2666743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:57.061285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d2051-b04f-4a63-aebd-deb5d2666743 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:57.061585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d2051-b04f-4a63-aebd-deb5d2666743 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:57.061861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d2051-b04f-4a63-aebd-deb5d2666743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:57.072591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d2051-b04f-4a63-aebd-deb5d2666743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:57.074062 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8d2051-b04f-4a63-aebd-deb5d2666743 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:57.084667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1241/2495] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:28:57.447121 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b9114-ae28-4a4a-9e93-a1452218dde5 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:57.448592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-641b9114-ae28-4a4a-9e93-a1452218dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:57.461021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b9114-ae28-4a4a-9e93-a1452218dde5 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:57.466279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:57.466709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:57.466990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:57.467286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] PATH_INFO: `/v3/projects/e17091ff17eb41388de7f3c2c97c2306` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:57.478446 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e17091ff17eb41388de7f3c2c97c2306)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:57.480897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:57.545705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e17091ff17eb41388de7f3c2c97c2306', 'request_id': 'req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:57.557006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e17091ff17eb41388de7f3c2c97c2306', 'request_id': 'req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:57.567533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e17091ff17eb41388de7f3c2c97c2306', 'request_id': 'req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:57.584704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] The token cache is being invalidate because project e17091ff17eb41388de7f3c2c97c2306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:28:57.585002 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:57.585711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-00d7ab9b-eb95-4b85-81bd-01dc2ad6db3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:57.587240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1255/2496] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:28:57 2022] DELETE /identity/v3/projects/e17091ff17eb41388de7f3c2c97c2306 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:57.592108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b13d7-95fa-4b89-a922-df459f884a9f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:57.592465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b13d7-95fa-4b89-a922-df459f884a9f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:57.592840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b13d7-95fa-4b89-a922-df459f884a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:57.613553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-147b13d7-95fa-4b89-a922-df459f884a9f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:57.615350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-147b13d7-95fa-4b89-a922-df459f884a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:57.632627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:57.691625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1242/2497] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:57.696834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d440c90-792d-4e9e-89a7-8b8b2dbfd5ef None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:57.696834 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d440c90-792d-4e9e-89a7-8b8b2dbfd5ef None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:57.697096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d440c90-792d-4e9e-89a7-8b8b2dbfd5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:57.719142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8d440c90-792d-4e9e-89a7-8b8b2dbfd5ef None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:57.720353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8d440c90-792d-4e9e-89a7-8b8b2dbfd5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:57.729387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1256/2498] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:57.733487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b13d7-95fa-4b89-a922-df459f884a9f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:57.743855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b13d7-95fa-4b89-a922-df459f884a9f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:57.748510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac12d9-a57a-4af6-b86a-21c639ac65cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:57.749702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac12d9-a57a-4af6-b86a-21c639ac65cd admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:57.749702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac12d9-a57a-4af6-b86a-21c639ac65cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:57.749923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac12d9-a57a-4af6-b86a-21c639ac65cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:57.750485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac12d9-a57a-4af6-b86a-21c639ac65cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:28:57.750787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac12d9-a57a-4af6-b86a-21c639ac65cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:57.752392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac12d9-a57a-4af6-b86a-21c639ac65cd admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:57.778089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1243/2499] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:28:57.781954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d440c90-792d-4e9e-89a7-8b8b2dbfd5ef None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:57.791937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d440c90-792d-4e9e-89a7-8b8b2dbfd5ef None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:57.796725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95880ac8-a118-4ed7-8cfa-d703cba66b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:57.797540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95880ac8-a118-4ed7-8cfa-d703cba66b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:57.797741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95880ac8-a118-4ed7-8cfa-d703cba66b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:57.798072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95880ac8-a118-4ed7-8cfa-d703cba66b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:57.798707 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95880ac8-a118-4ed7-8cfa-d703cba66b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:57.800616 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95880ac8-a118-4ed7-8cfa-d703cba66b31 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:57.831006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1257/2500] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:28:57 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:28:57.835046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac12d9-a57a-4af6-b86a-21c639ac65cd admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:57.845245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac12d9-a57a-4af6-b86a-21c639ac65cd admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:57.909222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda6e19-1675-4683-b8b0-fabb97914570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:57.910147 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda6e19-1675-4683-b8b0-fabb97914570 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:57.910986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda6e19-1675-4683-b8b0-fabb97914570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:57.911247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda6e19-1675-4683-b8b0-fabb97914570 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:57.911871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda6e19-1675-4683-b8b0-fabb97914570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:57.913972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda6e19-1675-4683-b8b0-fabb97914570 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:57.935461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1244/2501] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:28:57 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:28:57.939497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95880ac8-a118-4ed7-8cfa-d703cba66b31 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:57.950117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95880ac8-a118-4ed7-8cfa-d703cba66b31 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:57.955238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df1c8e-e4e4-4cb1-a9ba-0150073f53c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:57.955694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df1c8e-e4e4-4cb1-a9ba-0150073f53c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:57.955966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df1c8e-e4e4-4cb1-a9ba-0150073f53c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:57.956248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df1c8e-e4e4-4cb1-a9ba-0150073f53c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:57.957059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df1c8e-e4e4-4cb1-a9ba-0150073f53c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:57.959201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df1c8e-e4e4-4cb1-a9ba-0150073f53c4 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:57.969253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1258/2502] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:28:57.973887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda6e19-1675-4683-b8b0-fabb97914570 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:57.983732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda6e19-1675-4683-b8b0-fabb97914570 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:57.988329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbdc98-c4fb-42a9-a40b-f871e80fddce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:57.988961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbdc98-c4fb-42a9-a40b-f871e80fddce admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:57.989280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbdc98-c4fb-42a9-a40b-f871e80fddce admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:57.989581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbdc98-c4fb-42a9-a40b-f871e80fddce admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:57.990220 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbdc98-c4fb-42a9-a40b-f871e80fddce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:57.991789 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbdc98-c4fb-42a9-a40b-f871e80fddce admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:58.002916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1245/2503] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:28:58.006805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df1c8e-e4e4-4cb1-a9ba-0150073f53c4 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:58.016728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df1c8e-e4e4-4cb1-a9ba-0150073f53c4 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:58.021845 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162656a0-11d1-4117-b3dc-f820bbe18c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:58.022564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162656a0-11d1-4117-b3dc-f820bbe18c56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:58.022839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162656a0-11d1-4117-b3dc-f820bbe18c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:58.023106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162656a0-11d1-4117-b3dc-f820bbe18c56 admin admin] PATH_INFO: `/v3/projects/6cfeb479d8b44256b42158485748e9de/users/4ccd732bb94d469f86e2f6e9353584c3/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:58.038148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162656a0-11d1-4117-b3dc-f820bbe18c56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfeb479d8b44256b42158485748e9de, user_id=4ccd732bb94d469f86e2f6e9353584c3, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:58.040782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162656a0-11d1-4117-b3dc-f820bbe18c56 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:58.054272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1259/2504] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:28:58 2022] PUT /identity/v3/projects/6cfeb479d8b44256b42158485748e9de/users/4ccd732bb94d469f86e2f6e9353584c3/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:28:58.066903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9669f-41a8-465a-8403-d02e30ec4c5a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:58.067449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9669f-41a8-465a-8403-d02e30ec4c5a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:58.067718 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9669f-41a8-465a-8403-d02e30ec4c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:58.095442 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-35b9669f-41a8-465a-8403-d02e30ec4c5a None None] MFA Rules not processed for user `4ccd732bb94d469f86e2f6e9353584c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:58.096648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-35b9669f-41a8-465a-8403-d02e30ec4c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:58.163824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1246/2505] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:58.176635 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162656a0-11d1-4117-b3dc-f820bbe18c56 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:58.178514 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-162656a0-11d1-4117-b3dc-f820bbe18c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:58.200487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:58.238808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162656a0-11d1-4117-b3dc-f820bbe18c56 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:58.242440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c81254-44f7-4e9d-a0cd-9671f080ba0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:58.243042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c81254-44f7-4e9d-a0cd-9671f080ba0b service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:58.243318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c81254-44f7-4e9d-a0cd-9671f080ba0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:58.243588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c81254-44f7-4e9d-a0cd-9671f080ba0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:58.252367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c81254-44f7-4e9d-a0cd-9671f080ba0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:58.253819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c81254-44f7-4e9d-a0cd-9671f080ba0b service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:58.264367 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1260/2506] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:28:58.292285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9669f-41a8-465a-8403-d02e30ec4c5a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:58.311037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:28:58.362820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9669f-41a8-465a-8403-d02e30ec4c5a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:58.367774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:58.368429 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:58.368707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:58.368978 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] PATH_INFO: `/v3/users/4ccd732bb94d469f86e2f6e9353584c3` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:58.370146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ccd732bb94d469f86e2f6e9353584c3)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:58.372157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:58.423373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ccd732bb94d469f86e2f6e9353584c3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:58.424039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ccd732bb94d469f86e2f6e9353584c3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:58.431322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ccd732bb94d469f86e2f6e9353584c3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:58.439254 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ccd732bb94d469f86e2f6e9353584c3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:58.440093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ccd732bb94d469f86e2f6e9353584c3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:58.459679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1247/2507] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:28:58 2022] DELETE /identity/v3/users/4ccd732bb94d469f86e2f6e9353584c3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:28:58.464222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d58bbb-8f4e-435a-9c11-8803dff7e903 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:58.464506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d58bbb-8f4e-435a-9c11-8803dff7e903 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:58.464783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d58bbb-8f4e-435a-9c11-8803dff7e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:58.484822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-43d58bbb-8f4e-435a-9c11-8803dff7e903 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:28:58.486136 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-43d58bbb-8f4e-435a-9c11-8803dff7e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:58.550644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1261/2508] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:28:58.560997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:58.562639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:58.623284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3feded-1ea1-4451-97e9-29485c5f6b9c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:58.626907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77c5b4-93e2-478f-b481-9cae1e49dcc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:58.627516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77c5b4-93e2-478f-b481-9cae1e49dcc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:58.627793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77c5b4-93e2-478f-b481-9cae1e49dcc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:58.628059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77c5b4-93e2-478f-b481-9cae1e49dcc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:58.637469 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77c5b4-93e2-478f-b481-9cae1e49dcc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:58.638913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77c5b4-93e2-478f-b481-9cae1e49dcc2 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:58.649647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1248/2509] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:28:58.974747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d58bbb-8f4e-435a-9c11-8803dff7e903 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:28:58.976773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-43d58bbb-8f4e-435a-9c11-8803dff7e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:28:58.991676 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d58bbb-8f4e-435a-9c11-8803dff7e903 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:28:58.996357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:28:58.997073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:28:58.997544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:28:58.997939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] PATH_INFO: `/v3/projects/6cfeb479d8b44256b42158485748e9de` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:28:58.999070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfeb479d8b44256b42158485748e9de)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:28:59.001268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:28:59.063822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfeb479d8b44256b42158485748e9de', 'request_id': 'req-2536f512-1e37-4a8c-9426-7aa76bc83aad'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:59.073270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfeb479d8b44256b42158485748e9de', 'request_id': 'req-2536f512-1e37-4a8c-9426-7aa76bc83aad'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:59.080442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfeb479d8b44256b42158485748e9de', 'request_id': 'req-2536f512-1e37-4a8c-9426-7aa76bc83aad'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:59.093416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] The token cache is being invalidate because project 6cfeb479d8b44256b42158485748e9de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:28:59.093735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:59.094393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:28:59.095889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1262/2510] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:28:58 2022] DELETE /identity/v3/projects/6cfeb479d8b44256b42158485748e9de => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:02.324449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77c5b4-93e2-478f-b481-9cae1e49dcc2 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:02.325894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3a77c5b4-93e2-478f-b481-9cae1e49dcc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:02.337174 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77c5b4-93e2-478f-b481-9cae1e49dcc2 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:02.390109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:02.407647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f285b9e-4071-46bd-a6c1-2d0bfbfb584b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:02.408258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f285b9e-4071-46bd-a6c1-2d0bfbfb584b service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:02.408535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f285b9e-4071-46bd-a6c1-2d0bfbfb584b service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:02.409026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f285b9e-4071-46bd-a6c1-2d0bfbfb584b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:02.411275 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0f285b9e-4071-46bd-a6c1-2d0bfbfb584b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:02.475436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f285b9e-4071-46bd-a6c1-2d0bfbfb584b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:02.476853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f285b9e-4071-46bd-a6c1-2d0bfbfb584b service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:02.486628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1249/2511] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:06.040759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:06.042402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:06.056625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536f512-1e37-4a8c-9426-7aa76bc83aad admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:06.061650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:06.062387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.062781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.063150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] PATH_INFO: `/v3/users/c28866ff5b2a4dd4ba05270b369f064e` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.064364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28866ff5b2a4dd4ba05270b369f064e)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:06.066457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:06.117983 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:06.125024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28866ff5b2a4dd4ba05270b369f064e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.125820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28866ff5b2a4dd4ba05270b369f064e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.135323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28866ff5b2a4dd4ba05270b369f064e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.136106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28866ff5b2a4dd4ba05270b369f064e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.145877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6892c7d8-6b27-4742-bc94-81342e188c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28866ff5b2a4dd4ba05270b369f064e'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.169158 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1263/2512] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:06 2022] DELETE /identity/v3/users/c28866ff5b2a4dd4ba05270b369f064e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:06.343849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f285b9e-4071-46bd-a6c1-2d0bfbfb584b service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:06.356161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f285b9e-4071-46bd-a6c1-2d0bfbfb584b service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:06.416315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:06.416950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.417229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.417501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] PATH_INFO: `/v3/projects/45fc2abd2bae4264b1081c1673e3095e` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.418623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fc2abd2bae4264b1081c1673e3095e)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:06.420552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:06.465622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fc2abd2bae4264b1081c1673e3095e', 'request_id': 'req-2dceefd7-1c50-43bb-9767-537de4644565'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.472350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fc2abd2bae4264b1081c1673e3095e', 'request_id': 'req-2dceefd7-1c50-43bb-9767-537de4644565'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.478981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fc2abd2bae4264b1081c1673e3095e', 'request_id': 'req-2dceefd7-1c50-43bb-9767-537de4644565'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.489025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] The token cache is being invalidate because project 45fc2abd2bae4264b1081c1673e3095e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:06.489308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.489944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-2dceefd7-1c50-43bb-9767-537de4644565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:06.491411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1250/2513] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:06 2022] DELETE /identity/v3/projects/45fc2abd2bae4264b1081c1673e3095e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:06.496339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43b631-e6b1-4d34-b7c0-dbad6e2ff175 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.496615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43b631-e6b1-4d34-b7c0-dbad6e2ff175 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.496884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43b631-e6b1-4d34-b7c0-dbad6e2ff175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.516334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7e43b631-e6b1-4d34-b7c0-dbad6e2ff175 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:06.517556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7e43b631-e6b1-4d34-b7c0-dbad6e2ff175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:06.577672 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1264/2514] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:06.586940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9228f-6ced-47d5-8a1d-66c002a7fe39 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.587222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9228f-6ced-47d5-8a1d-66c002a7fe39 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.587495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9228f-6ced-47d5-8a1d-66c002a7fe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.606862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-b7c9228f-6ced-47d5-8a1d-66c002a7fe39 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:06.608046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b7c9228f-6ced-47d5-8a1d-66c002a7fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:06.615707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1251/2515] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:06.619904 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43b631-e6b1-4d34-b7c0-dbad6e2ff175 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:06.629866 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43b631-e6b1-4d34-b7c0-dbad6e2ff175 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:06.634492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe4205-f758-402e-9fa1-d77b8c4efa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:06.635084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe4205-f758-402e-9fa1-d77b8c4efa66 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.635359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe4205-f758-402e-9fa1-d77b8c4efa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.635626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe4205-f758-402e-9fa1-d77b8c4efa66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.636260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe4205-f758-402e-9fa1-d77b8c4efa66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:06.636556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe4205-f758-402e-9fa1-d77b8c4efa66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:06.638185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe4205-f758-402e-9fa1-d77b8c4efa66 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:06.650031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1265/2516] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:29:06.654512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9228f-6ced-47d5-8a1d-66c002a7fe39 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:06.664750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9228f-6ced-47d5-8a1d-66c002a7fe39 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:06.669344 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bf983-e729-4409-9775-b4990785acd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:06.669965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bf983-e729-4409-9775-b4990785acd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.670244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bf983-e729-4409-9775-b4990785acd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.670510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bf983-e729-4409-9775-b4990785acd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.671127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bf983-e729-4409-9775-b4990785acd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:06.673027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bf983-e729-4409-9775-b4990785acd2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:06.684117 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:06.702467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1252/2517] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:06 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:06.706653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe4205-f758-402e-9fa1-d77b8c4efa66 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:06.717043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe4205-f758-402e-9fa1-d77b8c4efa66 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:06.772966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd9e58-09e3-4da0-a6db-6eaaac9b17b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:06.773591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd9e58-09e3-4da0-a6db-6eaaac9b17b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.773865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd9e58-09e3-4da0-a6db-6eaaac9b17b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.774135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd9e58-09e3-4da0-a6db-6eaaac9b17b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.774772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd9e58-09e3-4da0-a6db-6eaaac9b17b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:06.776662 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd9e58-09e3-4da0-a6db-6eaaac9b17b4 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:06.806784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1266/2518] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:06 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:06.808010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bf983-e729-4409-9775-b4990785acd2 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:06.819706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bf983-e729-4409-9775-b4990785acd2 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:06.824582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb178048-1020-4e30-aa08-f07506760cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:06.825201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb178048-1020-4e30-aa08-f07506760cac admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.825647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb178048-1020-4e30-aa08-f07506760cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.825922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb178048-1020-4e30-aa08-f07506760cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.826574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb178048-1020-4e30-aa08-f07506760cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:06.828166 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb178048-1020-4e30-aa08-f07506760cac admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:06.842371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1253/2519] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:06.843507 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd9e58-09e3-4da0-a6db-6eaaac9b17b4 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:06.856384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd9e58-09e3-4da0-a6db-6eaaac9b17b4 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:06.861353 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c7f80-9d1c-4c51-8549-8fba673bf89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:06.862820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c7f80-9d1c-4c51-8549-8fba673bf89d admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.862820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c7f80-9d1c-4c51-8549-8fba673bf89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.863715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c7f80-9d1c-4c51-8549-8fba673bf89d admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.864598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c7f80-9d1c-4c51-8549-8fba673bf89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:06.866459 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c7f80-9d1c-4c51-8549-8fba673bf89d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:06.882832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1267/2520] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:06.886728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb178048-1020-4e30-aa08-f07506760cac admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:06.897883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb178048-1020-4e30-aa08-f07506760cac admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:06.902922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4985c18-5c00-404e-af95-d46fe864d04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:06.904284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4985c18-5c00-404e-af95-d46fe864d04a admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.904284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4985c18-5c00-404e-af95-d46fe864d04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.904284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4985c18-5c00-404e-af95-d46fe864d04a admin admin] PATH_INFO: `/v3/projects/4fb54ea7a0034a22a2a921af29326e85/users/97c9af8fba58431ea75a34fd6ed4af31/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.920762 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4985c18-5c00-404e-af95-d46fe864d04a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb54ea7a0034a22a2a921af29326e85, user_id=97c9af8fba58431ea75a34fd6ed4af31, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:06.923625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4985c18-5c00-404e-af95-d46fe864d04a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:06.937987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1254/2521] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:06 2022] PUT /identity/v3/projects/4fb54ea7a0034a22a2a921af29326e85/users/97c9af8fba58431ea75a34fd6ed4af31/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:06.949162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d500a-8abb-4b51-8202-97e169faef2e None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:06.949448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d500a-8abb-4b51-8202-97e169faef2e None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:06.949753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d500a-8abb-4b51-8202-97e169faef2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:06.978655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e29d500a-8abb-4b51-8202-97e169faef2e None None] MFA Rules not processed for user `97c9af8fba58431ea75a34fd6ed4af31`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:06.979843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e29d500a-8abb-4b51-8202-97e169faef2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:06.992110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:07.046687 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1268/2522] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:07.058683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4985c18-5c00-404e-af95-d46fe864d04a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:07.060545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c4985c18-5c00-404e-af95-d46fe864d04a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:07.136611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4985c18-5c00-404e-af95-d46fe864d04a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:07.142228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748d9d2-bf99-4808-9191-3d5e733abcdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:07.142836 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748d9d2-bf99-4808-9191-3d5e733abcdf service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:07.143113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748d9d2-bf99-4808-9191-3d5e733abcdf service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:07.143383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748d9d2-bf99-4808-9191-3d5e733abcdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:07.151345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748d9d2-bf99-4808-9191-3d5e733abcdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:07.152911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748d9d2-bf99-4808-9191-3d5e733abcdf service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:07.161946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1255/2523] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:07.459252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d500a-8abb-4b51-8202-97e169faef2e None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:07.524576 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d500a-8abb-4b51-8202-97e169faef2e None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:07.529311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:07.530065 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:07.530407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:07.530777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] PATH_INFO: `/v3/users/97c9af8fba58431ea75a34fd6ed4af31` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:07.531980 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c9af8fba58431ea75a34fd6ed4af31)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:07.534024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:07.586072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c9af8fba58431ea75a34fd6ed4af31'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:07.586733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c9af8fba58431ea75a34fd6ed4af31'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:07.594110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c9af8fba58431ea75a34fd6ed4af31'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:07.594772 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c9af8fba58431ea75a34fd6ed4af31'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:07.602720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c9af8fba58431ea75a34fd6ed4af31'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:07.621326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1269/2524] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:07 2022] DELETE /identity/v3/users/97c9af8fba58431ea75a34fd6ed4af31 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:07.627271 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8947a2-71af-4c0f-9cc1-d1fc9759b4b8 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:07.627783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8947a2-71af-4c0f-9cc1-d1fc9759b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:07.627902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8947a2-71af-4c0f-9cc1-d1fc9759b4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:07.648627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-ae8947a2-71af-4c0f-9cc1-d1fc9759b4b8 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:07.649918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ae8947a2-71af-4c0f-9cc1-d1fc9759b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:07.712119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1256/2525] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:07.721733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:07.723140 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:07.765604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:07.788810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd5d05-77be-4f66-a651-33c4c0e6b943 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:07.792698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88f974-7c31-4dc0-a070-67fbd61b6be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:07.793312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88f974-7c31-4dc0-a070-67fbd61b6be8 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:07.793625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88f974-7c31-4dc0-a070-67fbd61b6be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:07.793899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88f974-7c31-4dc0-a070-67fbd61b6be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:07.804274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88f974-7c31-4dc0-a070-67fbd61b6be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:07.805953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88f974-7c31-4dc0-a070-67fbd61b6be8 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:07.815642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1270/2526] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:08.091476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8947a2-71af-4c0f-9cc1-d1fc9759b4b8 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.093110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ae8947a2-71af-4c0f-9cc1-d1fc9759b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:08.107548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8947a2-71af-4c0f-9cc1-d1fc9759b4b8 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:08.112436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:08.113839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.114167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.114505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] PATH_INFO: `/v3/projects/4fb54ea7a0034a22a2a921af29326e85` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.115754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fb54ea7a0034a22a2a921af29326e85)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:08.117821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:08.161398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:08.172050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fb54ea7a0034a22a2a921af29326e85', 'request_id': 'req-cf08e7c6-814b-42a2-8229-c81984041319'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:08.183580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fb54ea7a0034a22a2a921af29326e85', 'request_id': 'req-cf08e7c6-814b-42a2-8229-c81984041319'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:08.190928 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fb54ea7a0034a22a2a921af29326e85', 'request_id': 'req-cf08e7c6-814b-42a2-8229-c81984041319'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:08.202659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] The token cache is being invalidate because project 4fb54ea7a0034a22a2a921af29326e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:08.203241 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:08.203904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf08e7c6-814b-42a2-8229-c81984041319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:08.205767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1257/2527] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:08 2022] DELETE /identity/v3/projects/4fb54ea7a0034a22a2a921af29326e85 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:08.210656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b3d18-d2fa-455d-b33c-6fb7b469a768 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.211161 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b3d18-d2fa-455d-b33c-6fb7b469a768 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.211370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b3d18-d2fa-455d-b33c-6fb7b469a768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.232401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ec9b3d18-d2fa-455d-b33c-6fb7b469a768 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:08.233785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ec9b3d18-d2fa-455d-b33c-6fb7b469a768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:08.293395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1271/2528] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:08.301876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bbd5a-95a7-4b34-8014-63fad2baaa34 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.302160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bbd5a-95a7-4b34-8014-63fad2baaa34 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.302431 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bbd5a-95a7-4b34-8014-63fad2baaa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.322144 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c47bbd5a-95a7-4b34-8014-63fad2baaa34 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:08.323334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c47bbd5a-95a7-4b34-8014-63fad2baaa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:08.333997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1258/2529] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:08.338134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b3d18-d2fa-455d-b33c-6fb7b469a768 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.349109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b3d18-d2fa-455d-b33c-6fb7b469a768 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:08.354680 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34fce43-fc43-4679-8529-29f57873103d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:08.354680 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fce43-fc43-4679-8529-29f57873103d admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.354848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fce43-fc43-4679-8529-29f57873103d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.355625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fce43-fc43-4679-8529-29f57873103d admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.355625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fce43-fc43-4679-8529-29f57873103d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:08.355921 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fce43-fc43-4679-8529-29f57873103d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:08.357558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34fce43-fc43-4679-8529-29f57873103d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:08.377221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1272/2530] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:29:08.380432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bbd5a-95a7-4b34-8014-63fad2baaa34 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.391291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bbd5a-95a7-4b34-8014-63fad2baaa34 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:08.397124 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b2beb-8392-434c-a027-a968a4c32d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:08.397860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b2beb-8392-434c-a027-a968a4c32d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.398236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b2beb-8392-434c-a027-a968a4c32d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.398633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b2beb-8392-434c-a027-a968a4c32d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.399439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b2beb-8392-434c-a027-a968a4c32d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:08.401459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b2beb-8392-434c-a027-a968a4c32d46 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:08.428842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d9515-299e-445a-a70a-233e368ebe4a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.429139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d9515-299e-445a-a70a-233e368ebe4a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.429412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d9515-299e-445a-a70a-233e368ebe4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.432545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1259/2531] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:08 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:08.436312 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b2beb-8392-434c-a027-a968a4c32d46 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.450308 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-0c5d9515-299e-445a-a70a-233e368ebe4a None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:08.451506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0c5d9515-299e-445a-a70a-233e368ebe4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:08.506146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b2beb-8392-434c-a027-a968a4c32d46 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:08.518422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b81975-c9f6-4c4c-bfce-b8a1cef0b5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:08.520319 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b81975-c9f6-4c4c-bfce-b8a1cef0b5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.520959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1273/2532] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 20:29:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:08.521085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b81975-c9f6-4c4c-bfce-b8a1cef0b5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.521085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b81975-c9f6-4c4c-bfce-b8a1cef0b5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.521882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b81975-c9f6-4c4c-bfce-b8a1cef0b5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:08.524397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b81975-c9f6-4c4c-bfce-b8a1cef0b5d9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:08.547407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d9515-299e-445a-a70a-233e368ebe4a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.551652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1260/2533] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:08 2022] POST /identity/v3/users => generated 282 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:08.551820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-0c5d9515-299e-445a-a70a-233e368ebe4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:08.553155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b81975-c9f6-4c4c-bfce-b8a1cef0b5d9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.563409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b81975-c9f6-4c4c-bfce-b8a1cef0b5d9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:08.569716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178bd2a-7fb8-492b-961c-1a7641e32d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:08.570635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178bd2a-7fb8-492b-961c-1a7641e32d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.570918 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178bd2a-7fb8-492b-961c-1a7641e32d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.571191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178bd2a-7fb8-492b-961c-1a7641e32d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.571876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178bd2a-7fb8-492b-961c-1a7641e32d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:08.573402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178bd2a-7fb8-492b-961c-1a7641e32d73 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:08.585363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1261/2534] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:08.589760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178bd2a-7fb8-492b-961c-1a7641e32d73 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.601233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178bd2a-7fb8-492b-961c-1a7641e32d73 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:08.606655 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc9e9d-c56f-4f31-99c5-5e3674f1f05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:08.607335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc9e9d-c56f-4f31-99c5-5e3674f1f05e admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.607611 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc9e9d-c56f-4f31-99c5-5e3674f1f05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.607880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc9e9d-c56f-4f31-99c5-5e3674f1f05e admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.608516 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc9e9d-c56f-4f31-99c5-5e3674f1f05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:08.610130 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc9e9d-c56f-4f31-99c5-5e3674f1f05e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:08.626217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1262/2535] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:08.630155 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d9515-299e-445a-a70a-233e368ebe4a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:08.630456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc9e9d-c56f-4f31-99c5-5e3674f1f05e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.636152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0dc10f-3bd1-4421-aac4-730b9bbc94ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:08.636843 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dc10f-3bd1-4421-aac4-730b9bbc94ae service swift] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.637203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dc10f-3bd1-4421-aac4-730b9bbc94ae service swift] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.637462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0dc10f-3bd1-4421-aac4-730b9bbc94ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.644103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc9e9d-c56f-4f31-99c5-5e3674f1f05e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:08.649272 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0dc10f-3bd1-4421-aac4-730b9bbc94ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:08.653502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0dc10f-3bd1-4421-aac4-730b9bbc94ae service swift] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:08.656118 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8694762-5db6-4981-bc98-6d48746bad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:08.656795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8694762-5db6-4981-bc98-6d48746bad48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.657066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8694762-5db6-4981-bc98-6d48746bad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.657406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8694762-5db6-4981-bc98-6d48746bad48 admin admin] PATH_INFO: `/v3/projects/15054958c1dc42f09f4b601e87b5a8b9/users/5326995d46a040aa8e11b4fd31a30dee/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.657827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:08.662453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1274/2536] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:29:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 05 20:29:08.674701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8694762-5db6-4981-bc98-6d48746bad48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15054958c1dc42f09f4b601e87b5a8b9, user_id=5326995d46a040aa8e11b4fd31a30dee, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:08.677717 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8694762-5db6-4981-bc98-6d48746bad48 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:08.692689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1263/2537] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:08 2022] PUT /identity/v3/projects/15054958c1dc42f09f4b601e87b5a8b9/users/5326995d46a040aa8e11b4fd31a30dee/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:08.704298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c6d8d-32a4-4c2e-a1a1-160187392041 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.704972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c6d8d-32a4-4c2e-a1a1-160187392041 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.705360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c6d8d-32a4-4c2e-a1a1-160187392041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.738045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-343c6d8d-32a4-4c2e-a1a1-160187392041 None None] MFA Rules not processed for user `5326995d46a040aa8e11b4fd31a30dee`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:08.739339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-343c6d8d-32a4-4c2e-a1a1-160187392041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:08.817561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1275/2538] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:08.819736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8694762-5db6-4981-bc98-6d48746bad48 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.868986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:08.895367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8694762-5db6-4981-bc98-6d48746bad48 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:08.901306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c53b1-5325-46c7-a141-680daf6cd44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:08.901953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c53b1-5325-46c7-a141-680daf6cd44a admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:08.902236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c53b1-5325-46c7-a141-680daf6cd44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:08.902521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c53b1-5325-46c7-a141-680daf6cd44a admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:08.903143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c53b1-5325-46c7-a141-680daf6cd44a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:08.905049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c53b1-5325-46c7-a141-680daf6cd44a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:08.934697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1264/2539] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:08 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:08.939157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c6d8d-32a4-4c2e-a1a1-160187392041 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:08.951153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c6d8d-32a4-4c2e-a1a1-160187392041 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:09.008588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae025adc-050b-48e8-a527-528578d5a534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:09.009190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae025adc-050b-48e8-a527-528578d5a534 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:09.009471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae025adc-050b-48e8-a527-528578d5a534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:09.009767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae025adc-050b-48e8-a527-528578d5a534 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:09.010385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae025adc-050b-48e8-a527-528578d5a534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:09.012321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae025adc-050b-48e8-a527-528578d5a534 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:09.035072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1276/2540] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:08 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:09.039279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c53b1-5325-46c7-a141-680daf6cd44a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:09.049904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c53b1-5325-46c7-a141-680daf6cd44a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:09.054468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6d310-3242-4f4d-8611-8de7b83af48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:09.055078 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6d310-3242-4f4d-8611-8de7b83af48f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:09.055338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6d310-3242-4f4d-8611-8de7b83af48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:09.055607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6d310-3242-4f4d-8611-8de7b83af48f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:09.056240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6d310-3242-4f4d-8611-8de7b83af48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:09.057880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6d310-3242-4f4d-8611-8de7b83af48f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:09.073425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1265/2541] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:09.075402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae025adc-050b-48e8-a527-528578d5a534 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:09.085842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae025adc-050b-48e8-a527-528578d5a534 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:09.090414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50272dd-87b4-4d73-960a-c76062890ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:09.091061 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50272dd-87b4-4d73-960a-c76062890ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:09.091339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50272dd-87b4-4d73-960a-c76062890ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:09.091615 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50272dd-87b4-4d73-960a-c76062890ad8 admin admin] PATH_INFO: `/v3/projects/99285ac4a43e4bc6aeebbf3b63f5bfc7/users/92878395242d42f4a320b14b2d016738/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:09.107788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50272dd-87b4-4d73-960a-c76062890ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99285ac4a43e4bc6aeebbf3b63f5bfc7, user_id=92878395242d42f4a320b14b2d016738, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:09.110547 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50272dd-87b4-4d73-960a-c76062890ad8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:09.124628 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1277/2542] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:09 2022] PUT /identity/v3/projects/99285ac4a43e4bc6aeebbf3b63f5bfc7/users/92878395242d42f4a320b14b2d016738/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:09.128674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6d310-3242-4f4d-8611-8de7b83af48f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:09.139132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6d310-3242-4f4d-8611-8de7b83af48f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:09.196682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9895ca-1e27-4806-b5f1-fcddaa2f4387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:09.197291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9895ca-1e27-4806-b5f1-fcddaa2f4387 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:09.197592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9895ca-1e27-4806-b5f1-fcddaa2f4387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:09.197863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9895ca-1e27-4806-b5f1-fcddaa2f4387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:09.198504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9895ca-1e27-4806-b5f1-fcddaa2f4387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:09.200088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9895ca-1e27-4806-b5f1-fcddaa2f4387 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:09.210030 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1266/2543] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:09.214304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50272dd-87b4-4d73-960a-c76062890ad8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:09.224848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50272dd-87b4-4d73-960a-c76062890ad8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:09.229454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99337db8-3cbe-4938-b249-43ff87439437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:09.230135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99337db8-3cbe-4938-b249-43ff87439437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:09.230413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99337db8-3cbe-4938-b249-43ff87439437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:09.230682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99337db8-3cbe-4938-b249-43ff87439437 admin admin] PATH_INFO: `/v3/projects/99285ac4a43e4bc6aeebbf3b63f5bfc7/users/92878395242d42f4a320b14b2d016738/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:09.232577 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99337db8-3cbe-4938-b249-43ff87439437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99285ac4a43e4bc6aeebbf3b63f5bfc7, user_id=92878395242d42f4a320b14b2d016738, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:09.235243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99337db8-3cbe-4938-b249-43ff87439437 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:09.246112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1278/2544] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:09 2022] PUT /identity/v3/projects/99285ac4a43e4bc6aeebbf3b63f5bfc7/users/92878395242d42f4a320b14b2d016738/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:09.257189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28e4ec-5f2a-424d-99ad-c12491a32b77 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:09.257473 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28e4ec-5f2a-424d-99ad-c12491a32b77 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:09.257901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28e4ec-5f2a-424d-99ad-c12491a32b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:09.285326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-5f28e4ec-5f2a-424d-99ad-c12491a32b77 None None] MFA Rules not processed for user `92878395242d42f4a320b14b2d016738`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:09.286580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5f28e4ec-5f2a-424d-99ad-c12491a32b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:09.357313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1267/2545] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:09.369305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99337db8-3cbe-4938-b249-43ff87439437 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:09.371099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-99337db8-3cbe-4938-b249-43ff87439437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:09.417890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:09.433732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99337db8-3cbe-4938-b249-43ff87439437 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:09.437690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d774d0-dafa-43bc-a76e-fd5ecd6671d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:09.438303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d774d0-dafa-43bc-a76e-fd5ecd6671d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:09.438582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d774d0-dafa-43bc-a76e-fd5ecd6671d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:09.438851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d774d0-dafa-43bc-a76e-fd5ecd6671d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:09.448959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d774d0-dafa-43bc-a76e-fd5ecd6671d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:09.450433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d774d0-dafa-43bc-a76e-fd5ecd6671d4 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:09.460040 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1279/2546] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:09.894215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28e4ec-5f2a-424d-99ad-c12491a32b77 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:09.959896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:09.984576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28e4ec-5f2a-424d-99ad-c12491a32b77 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:09.991513 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:09.993304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:09.993787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:09.994213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] PATH_INFO: `/v3/users/5326995d46a040aa8e11b4fd31a30dee` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:09.996179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5326995d46a040aa8e11b4fd31a30dee)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:09.998538 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:10.062340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5326995d46a040aa8e11b4fd31a30dee'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.063058 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5326995d46a040aa8e11b4fd31a30dee'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.072106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5326995d46a040aa8e11b4fd31a30dee'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.081395 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5326995d46a040aa8e11b4fd31a30dee'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.082179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5326995d46a040aa8e11b4fd31a30dee'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.105977 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1268/2547] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:09 2022] DELETE /identity/v3/users/5326995d46a040aa8e11b4fd31a30dee => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:10.111612 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b624-b571-47d6-b469-568474af53d4 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:10.111900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b624-b571-47d6-b469-568474af53d4 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:10.112174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b624-b571-47d6-b469-568474af53d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:10.142221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1370b624-b571-47d6-b469-568474af53d4 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:10.143460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1370b624-b571-47d6-b469-568474af53d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:10.208503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1280/2548] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:10.218446 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:10.219882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:10.282245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b0b3c-cf9a-4a7e-8afa-4a60f9d617d5 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:10.287081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c0688-ff50-48cc-b592-a953e6c62b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:10.288324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c0688-ff50-48cc-b592-a953e6c62b35 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:10.288620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c0688-ff50-48cc-b592-a953e6c62b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:10.288895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c0688-ff50-48cc-b592-a953e6c62b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:10.299063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c0688-ff50-48cc-b592-a953e6c62b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:10.300570 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c0688-ff50-48cc-b592-a953e6c62b35 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:10.312185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1269/2549] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:10.604939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370b624-b571-47d6-b469-568474af53d4 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:10.606406 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1370b624-b571-47d6-b469-568474af53d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:10.626134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370b624-b571-47d6-b469-568474af53d4 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:10.631789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:10.632641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:10.633147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:10.633805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] PATH_INFO: `/v3/projects/15054958c1dc42f09f4b601e87b5a8b9` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:10.635017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15054958c1dc42f09f4b601e87b5a8b9)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:10.637317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:10.686781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15054958c1dc42f09f4b601e87b5a8b9', 'request_id': 'req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.694700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15054958c1dc42f09f4b601e87b5a8b9', 'request_id': 'req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.701114 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15054958c1dc42f09f4b601e87b5a8b9', 'request_id': 'req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.712429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] The token cache is being invalidate because project 15054958c1dc42f09f4b601e87b5a8b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:10.712725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.713414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.714940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1281/2550] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:10 2022] DELETE /identity/v3/projects/15054958c1dc42f09f4b601e87b5a8b9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:10.718951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c0688-ff50-48cc-b592-a953e6c62b35 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:10.730884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c0688-ff50-48cc-b592-a953e6c62b35 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:10.782947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:10.791153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:10.792363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:10.793162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:10.793579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] PATH_INFO: `/v3/users/92878395242d42f4a320b14b2d016738` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:10.794955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92878395242d42f4a320b14b2d016738)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:10.798540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:10.857981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92878395242d42f4a320b14b2d016738'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.858943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92878395242d42f4a320b14b2d016738'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.873631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92878395242d42f4a320b14b2d016738'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.885299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92878395242d42f4a320b14b2d016738'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.886420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d7a2795c-3d5d-4678-be2c-aa7d8f3fe314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92878395242d42f4a320b14b2d016738'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:10.906894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1270/2551] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:10 2022] DELETE /identity/v3/users/92878395242d42f4a320b14b2d016738 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:11.186225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:11.188088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:11.216997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb8303-fd66-44e5-8ad9-ed93ed9a8e00 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:11.327297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:11.328004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:11.328289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:11.329334 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] PATH_INFO: `/v3/projects/99285ac4a43e4bc6aeebbf3b63f5bfc7` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:11.330540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99285ac4a43e4bc6aeebbf3b63f5bfc7)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:11.333352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:11.346405 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:11.419055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99285ac4a43e4bc6aeebbf3b63f5bfc7', 'request_id': 'req-482e0634-2153-4e4e-807d-61cfd3442716'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:11.443381 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99285ac4a43e4bc6aeebbf3b63f5bfc7', 'request_id': 'req-482e0634-2153-4e4e-807d-61cfd3442716'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:11.455319 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99285ac4a43e4bc6aeebbf3b63f5bfc7', 'request_id': 'req-482e0634-2153-4e4e-807d-61cfd3442716'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:11.468765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] The token cache is being invalidate because project 99285ac4a43e4bc6aeebbf3b63f5bfc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:11.469057 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:11.469755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-482e0634-2153-4e4e-807d-61cfd3442716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:11.474659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1282/2552] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:11 2022] DELETE /identity/v3/projects/99285ac4a43e4bc6aeebbf3b63f5bfc7 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:11.476481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d872b3-75d3-401b-b3a5-f10e1b91bffb None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:11.477617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d872b3-75d3-401b-b3a5-f10e1b91bffb None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:11.478055 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d872b3-75d3-401b-b3a5-f10e1b91bffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:11.502424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-c9d872b3-75d3-401b-b3a5-f10e1b91bffb None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:11.504613 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-c9d872b3-75d3-401b-b3a5-f10e1b91bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:11.569295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1271/2553] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:11.578328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc228f-fb7e-4b41-badd-2a4383045045 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:11.578613 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc228f-fb7e-4b41-badd-2a4383045045 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:11.578885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc228f-fb7e-4b41-badd-2a4383045045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:11.600280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-a6dc228f-fb7e-4b41-badd-2a4383045045 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:11.601556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a6dc228f-fb7e-4b41-badd-2a4383045045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:11.611290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1283/2554] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:11.616687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d872b3-75d3-401b-b3a5-f10e1b91bffb None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:11.627256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d872b3-75d3-401b-b3a5-f10e1b91bffb None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:11.632975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf9172-7067-46eb-9978-2e59c3545633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:11.634682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf9172-7067-46eb-9978-2e59c3545633 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:11.634682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf9172-7067-46eb-9978-2e59c3545633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:11.634682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf9172-7067-46eb-9978-2e59c3545633 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:11.635069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf9172-7067-46eb-9978-2e59c3545633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:11.635463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf9172-7067-46eb-9978-2e59c3545633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:11.637134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf9172-7067-46eb-9978-2e59c3545633 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:11.657023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1272/2555] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:29:11.659678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc228f-fb7e-4b41-badd-2a4383045045 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:11.670618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc228f-fb7e-4b41-badd-2a4383045045 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:11.676714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26456997-06d0-4c96-a423-e1f1809a11d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:11.677919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26456997-06d0-4c96-a423-e1f1809a11d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:11.678088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26456997-06d0-4c96-a423-e1f1809a11d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:11.678574 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26456997-06d0-4c96-a423-e1f1809a11d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:11.679576 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26456997-06d0-4c96-a423-e1f1809a11d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:11.682071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26456997-06d0-4c96-a423-e1f1809a11d5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:11.713619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1284/2556] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:11 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:11.717462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf9172-7067-46eb-9978-2e59c3545633 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:11.728542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf9172-7067-46eb-9978-2e59c3545633 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:11.813953 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d526265-93f6-456b-b49c-2c3ceb62cec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:11.814586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d526265-93f6-456b-b49c-2c3ceb62cec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:11.814869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d526265-93f6-456b-b49c-2c3ceb62cec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:11.815728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d526265-93f6-456b-b49c-2c3ceb62cec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:11.816425 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d526265-93f6-456b-b49c-2c3ceb62cec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:11.818693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d526265-93f6-456b-b49c-2c3ceb62cec3 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:11.840706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:11.879186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1273/2557] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:11 2022] POST /identity/v3/users => generated 279 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:11.883290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26456997-06d0-4c96-a423-e1f1809a11d5 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:11.899870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26456997-06d0-4c96-a423-e1f1809a11d5 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:11.911509 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee7ede-f84d-407a-9200-7773f68e1982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:11.912291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee7ede-f84d-407a-9200-7773f68e1982 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:11.912815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee7ede-f84d-407a-9200-7773f68e1982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:11.913143 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee7ede-f84d-407a-9200-7773f68e1982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:11.914021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee7ede-f84d-407a-9200-7773f68e1982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:11.916189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee7ede-f84d-407a-9200-7773f68e1982 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:11.929095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1285/2558] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:11.932712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d526265-93f6-456b-b49c-2c3ceb62cec3 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:11.964139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d526265-93f6-456b-b49c-2c3ceb62cec3 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:11.971113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012daac-2056-4d43-a131-5b7dbfbe43cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:11.974198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012daac-2056-4d43-a131-5b7dbfbe43cb admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:11.974678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012daac-2056-4d43-a131-5b7dbfbe43cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:11.975171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012daac-2056-4d43-a131-5b7dbfbe43cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:11.976245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012daac-2056-4d43-a131-5b7dbfbe43cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:11.978354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012daac-2056-4d43-a131-5b7dbfbe43cb admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:11.998834 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1274/2559] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:12.005036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee7ede-f84d-407a-9200-7773f68e1982 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:12.026522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee7ede-f84d-407a-9200-7773f68e1982 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:12.035742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f9711-61b7-48b1-8642-0bf9eff3c539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:12.036660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9711-61b7-48b1-8642-0bf9eff3c539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:12.036660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9711-61b7-48b1-8642-0bf9eff3c539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:12.036939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9711-61b7-48b1-8642-0bf9eff3c539 admin admin] PATH_INFO: `/v3/projects/64da4b58571040509d92ef27a0b3223f/users/fe1ed657cc3f4dcb80f1559adfee5c7f/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:12.059910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f9711-61b7-48b1-8642-0bf9eff3c539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64da4b58571040509d92ef27a0b3223f, user_id=fe1ed657cc3f4dcb80f1559adfee5c7f, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:12.064777 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f9711-61b7-48b1-8642-0bf9eff3c539 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:12.088932 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1286/2560] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:12 2022] PUT /identity/v3/projects/64da4b58571040509d92ef27a0b3223f/users/fe1ed657cc3f4dcb80f1559adfee5c7f/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:12.099115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efa557-2192-4d00-8f11-77cf60cfe0d8 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:12.099299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efa557-2192-4d00-8f11-77cf60cfe0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:12.099576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efa557-2192-4d00-8f11-77cf60cfe0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:12.136714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-90efa557-2192-4d00-8f11-77cf60cfe0d8 None None] MFA Rules not processed for user `fe1ed657cc3f4dcb80f1559adfee5c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:12.138151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-90efa557-2192-4d00-8f11-77cf60cfe0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:12.226597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1275/2561] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:12.231066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f9711-61b7-48b1-8642-0bf9eff3c539 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:12.334167 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f9711-61b7-48b1-8642-0bf9eff3c539 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:12.339186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d43cd-e205-4419-a785-2e97a540044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:12.339967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d43cd-e205-4419-a785-2e97a540044e admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:12.340247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d43cd-e205-4419-a785-2e97a540044e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:12.340558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d43cd-e205-4419-a785-2e97a540044e admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:12.341211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d43cd-e205-4419-a785-2e97a540044e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:12.343248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d43cd-e205-4419-a785-2e97a540044e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:12.375858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1287/2562] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:12 2022] POST /identity/v3/projects => generated 357 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:12.380217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efa557-2192-4d00-8f11-77cf60cfe0d8 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:12.392625 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efa557-2192-4d00-8f11-77cf60cfe0d8 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:12.493942 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1eb29-8399-4876-8df4-603f8dcf8a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:12.494696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1eb29-8399-4876-8df4-603f8dcf8a2e admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:12.494987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1eb29-8399-4876-8df4-603f8dcf8a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:12.495315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1eb29-8399-4876-8df4-603f8dcf8a2e admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:12.496021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1eb29-8399-4876-8df4-603f8dcf8a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:12.499688 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1eb29-8399-4876-8df4-603f8dcf8a2e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:12.535343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1276/2563] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:12 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:12.538759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d43cd-e205-4419-a785-2e97a540044e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:12.556781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d43cd-e205-4419-a785-2e97a540044e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:12.564387 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71238e3c-7337-4d1b-8f01-98db3b52a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:12.565325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238e3c-7337-4d1b-8f01-98db3b52a07c admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:12.565728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238e3c-7337-4d1b-8f01-98db3b52a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:12.566304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238e3c-7337-4d1b-8f01-98db3b52a07c admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:12.567084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71238e3c-7337-4d1b-8f01-98db3b52a07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:12.569178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71238e3c-7337-4d1b-8f01-98db3b52a07c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:12.583238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1288/2564] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:12.586985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1eb29-8399-4876-8df4-603f8dcf8a2e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:12.599578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1eb29-8399-4876-8df4-603f8dcf8a2e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:12.610782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5686b55-9203-4a5c-b965-fc433dc858ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:12.612072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5686b55-9203-4a5c-b965-fc433dc858ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:12.612300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5686b55-9203-4a5c-b965-fc433dc858ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:12.612577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5686b55-9203-4a5c-b965-fc433dc858ac admin admin] PATH_INFO: `/v3/projects/48da174e1b1247519cd78caa1de305d2/users/8b70da2510f64a50bd6ea94a626d78a3/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:12.636346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5686b55-9203-4a5c-b965-fc433dc858ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=48da174e1b1247519cd78caa1de305d2, user_id=8b70da2510f64a50bd6ea94a626d78a3, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:12.639566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5686b55-9203-4a5c-b965-fc433dc858ac admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:12.662472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1277/2565] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:12 2022] PUT /identity/v3/projects/48da174e1b1247519cd78caa1de305d2/users/8b70da2510f64a50bd6ea94a626d78a3/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:12.666227 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71238e3c-7337-4d1b-8f01-98db3b52a07c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:12.690050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71238e3c-7337-4d1b-8f01-98db3b52a07c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:12.696118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:12.778234 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d071c-500a-413e-b7f7-8a567bc38e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:12.779346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d071c-500a-413e-b7f7-8a567bc38e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:12.780018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d071c-500a-413e-b7f7-8a567bc38e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:12.780520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d071c-500a-413e-b7f7-8a567bc38e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:12.785886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d071c-500a-413e-b7f7-8a567bc38e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:12.787865 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d071c-500a-413e-b7f7-8a567bc38e41 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:12.801287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1289/2566] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:12.805224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5686b55-9203-4a5c-b965-fc433dc858ac admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:12.825420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5686b55-9203-4a5c-b965-fc433dc858ac admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:12.837198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b4bd7-f7f0-4eb8-b8d7-7c11ef755117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:12.838402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b4bd7-f7f0-4eb8-b8d7-7c11ef755117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:12.838774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b4bd7-f7f0-4eb8-b8d7-7c11ef755117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:12.839129 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b4bd7-f7f0-4eb8-b8d7-7c11ef755117 admin admin] PATH_INFO: `/v3/projects/48da174e1b1247519cd78caa1de305d2/users/8b70da2510f64a50bd6ea94a626d78a3/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:12.842020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b4bd7-f7f0-4eb8-b8d7-7c11ef755117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48da174e1b1247519cd78caa1de305d2, user_id=8b70da2510f64a50bd6ea94a626d78a3, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:12.846049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b4bd7-f7f0-4eb8-b8d7-7c11ef755117 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:12.864263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1278/2567] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:12 2022] PUT /identity/v3/projects/48da174e1b1247519cd78caa1de305d2/users/8b70da2510f64a50bd6ea94a626d78a3/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:12.880178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5ba1d-698b-4b6f-9ac6-233631c6720b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:12.880546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5ba1d-698b-4b6f-9ac6-233631c6720b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:12.880909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5ba1d-698b-4b6f-9ac6-233631c6720b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:12.913653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-9ee5ba1d-698b-4b6f-9ac6-233631c6720b None None] MFA Rules not processed for user `8b70da2510f64a50bd6ea94a626d78a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:12.915412 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9ee5ba1d-698b-4b6f-9ac6-233631c6720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:13.003024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1290/2568] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:13.015608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b4bd7-f7f0-4eb8-b8d7-7c11ef755117 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:13.017589 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b42b4bd7-f7f0-4eb8-b8d7-7c11ef755117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:13.051897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:13.087537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b4bd7-f7f0-4eb8-b8d7-7c11ef755117 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:13.091330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cb010-62c0-4e2e-9773-a02a55d56851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:13.091956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cb010-62c0-4e2e-9773-a02a55d56851 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:13.092293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cb010-62c0-4e2e-9773-a02a55d56851 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:13.092628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cb010-62c0-4e2e-9773-a02a55d56851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:13.102644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cb010-62c0-4e2e-9773-a02a55d56851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:13.104160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cb010-62c0-4e2e-9773-a02a55d56851 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:13.116639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1279/2569] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:13 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:13.246090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5ba1d-698b-4b6f-9ac6-233631c6720b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:13.264787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5ba1d-698b-4b6f-9ac6-233631c6720b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:13.269978 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063296db-4b49-4cbb-902d-0a15e7f3a626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:13.271222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063296db-4b49-4cbb-902d-0a15e7f3a626 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:13.271222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063296db-4b49-4cbb-902d-0a15e7f3a626 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:13.271827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063296db-4b49-4cbb-902d-0a15e7f3a626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:13.370436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063296db-4b49-4cbb-902d-0a15e7f3a626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:13.372117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063296db-4b49-4cbb-902d-0a15e7f3a626 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:13.381876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1291/2570] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:13.810532 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cb010-62c0-4e2e-9773-a02a55d56851 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:13.812521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-161cb010-62c0-4e2e-9773-a02a55d56851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:13.910964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cb010-62c0-4e2e-9773-a02a55d56851 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:13.916313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:13.916934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:13.917212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:13.917483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:13.919308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:13.932913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:13.934515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:13.948714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1280/2571] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:16.773528 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063296db-4b49-4cbb-902d-0a15e7f3a626 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:16.774966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-063296db-4b49-4cbb-902d-0a15e7f3a626 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:16.807284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:16.834102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063296db-4b49-4cbb-902d-0a15e7f3a626 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:16.837583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61401b22-6ac7-417b-b613-41afba622ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:16.838188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61401b22-6ac7-417b-b613-41afba622ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:16.838458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61401b22-6ac7-417b-b613-41afba622ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:16.838721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61401b22-6ac7-417b-b613-41afba622ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:16.840622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-61401b22-6ac7-417b-b613-41afba622ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:16.905805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61401b22-6ac7-417b-b613-41afba622ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:16.907502 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61401b22-6ac7-417b-b613-41afba622ca8 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:16.920126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1292/2572] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:17.148860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:17.150529 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:17.168202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553f82c-6f24-4bc0-a621-fde8439b987d service placement] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:17.173008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:17.173708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:17.173996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:17.174272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] PATH_INFO: `/v3/users/5500db61c95f4245945784ae6b13f197` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:17.176127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5500db61c95f4245945784ae6b13f197)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:17.178200 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:17.241372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5500db61c95f4245945784ae6b13f197'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.242092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5500db61c95f4245945784ae6b13f197'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.250284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5500db61c95f4245945784ae6b13f197'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.260898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5500db61c95f4245945784ae6b13f197'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.261559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5500db61c95f4245945784ae6b13f197'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.280336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1281/2573] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:17 2022] DELETE /identity/v3/users/5500db61c95f4245945784ae6b13f197 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:17.533157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61401b22-6ac7-417b-b613-41afba622ca8 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:17.544252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61401b22-6ac7-417b-b613-41afba622ca8 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:17.613964 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:17.614671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:17.614924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:17.615190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] PATH_INFO: `/v3/projects/5d5f5bc40cdc4be792aff778d3020c6d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:17.616482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d5f5bc40cdc4be792aff778d3020c6d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:17.619067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:17.690132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d5f5bc40cdc4be792aff778d3020c6d', 'request_id': 'req-d5f7cef3-9b44-4809-8862-c97b1c6cab70'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.702785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d5f5bc40cdc4be792aff778d3020c6d', 'request_id': 'req-d5f7cef3-9b44-4809-8862-c97b1c6cab70'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.711539 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d5f5bc40cdc4be792aff778d3020c6d', 'request_id': 'req-d5f7cef3-9b44-4809-8862-c97b1c6cab70'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.724448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] The token cache is being invalidate because project 5d5f5bc40cdc4be792aff778d3020c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:17.724749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.725382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.727120 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1293/2574] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:17 2022] DELETE /identity/v3/projects/5d5f5bc40cdc4be792aff778d3020c6d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:17.731818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:17.732731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:17.748380 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e63974-b82e-4d00-b5ee-b6ce43a44803 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:17.778220 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:17.819463 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:17.820187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:17.820464 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:17.820736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] PATH_INFO: `/v3/users/858e41026fd1418c8ba3c725d12de488` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:17.822110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=858e41026fd1418c8ba3c725d12de488)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:17.837171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:17.877211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '858e41026fd1418c8ba3c725d12de488'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.878949 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '858e41026fd1418c8ba3c725d12de488'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.885678 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '858e41026fd1418c8ba3c725d12de488'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.893841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '858e41026fd1418c8ba3c725d12de488'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.895115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '858e41026fd1418c8ba3c725d12de488'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:17.916841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1282/2575] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:17 2022] DELETE /identity/v3/users/858e41026fd1418c8ba3c725d12de488 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:18.114565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:18.116385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:18.133008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7cef3-9b44-4809-8862-c97b1c6cab70 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:18.137914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:18.208178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:18.208819 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:18.209098 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:18.209370 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] PATH_INFO: `/v3/projects/b4002f10776f48c5a34aaa62ae41dd60` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:18.210674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4002f10776f48c5a34aaa62ae41dd60)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:18.212586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:18.261870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4002f10776f48c5a34aaa62ae41dd60', 'request_id': 'req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:18.270047 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4002f10776f48c5a34aaa62ae41dd60', 'request_id': 'req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:18.276372 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4002f10776f48c5a34aaa62ae41dd60', 'request_id': 'req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:18.287247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] The token cache is being invalidate because project b4002f10776f48c5a34aaa62ae41dd60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:18.287532 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:18.288265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:18.289884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1294/2576] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:18 2022] DELETE /identity/v3/projects/b4002f10776f48c5a34aaa62ae41dd60 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:20.436694 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:20.438681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:20.459698 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7678114-4181-43d7-b959-1fbe0277a395 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:20.533547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:20.533547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:20.533547 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:20.534085 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] PATH_INFO: `/v3/users/ff9b14ab630d4a928614c60e6aea667c` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:20.540555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff9b14ab630d4a928614c60e6aea667c)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:20.542575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:20.612050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff9b14ab630d4a928614c60e6aea667c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:20.615245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff9b14ab630d4a928614c60e6aea667c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:20.617759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:20.624559 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff9b14ab630d4a928614c60e6aea667c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:20.632921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff9b14ab630d4a928614c60e6aea667c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:20.633776 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dc7702ac-930c-4d34-b8ac-905d765ed3da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff9b14ab630d4a928614c60e6aea667c'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:20.653239 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1283/2577] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:20 2022] DELETE /identity/v3/users/ff9b14ab630d4a928614c60e6aea667c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:20.829213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:20.830677 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:20.845035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca7099-ecd0-4adc-9cad-3fc3ca7c5261 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:20.919457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:20.920177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:20.920526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:20.920863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] PATH_INFO: `/v3/projects/140a5d8a1d264d8bad17d352591fa6a2` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:20.922097 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=140a5d8a1d264d8bad17d352591fa6a2)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:20.924099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:20.968035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:20.976317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '140a5d8a1d264d8bad17d352591fa6a2', 'request_id': 'req-13cfd2b8-8002-474b-805b-c877e1d4e868'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:20.983480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '140a5d8a1d264d8bad17d352591fa6a2', 'request_id': 'req-13cfd2b8-8002-474b-805b-c877e1d4e868'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:20.989592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '140a5d8a1d264d8bad17d352591fa6a2', 'request_id': 'req-13cfd2b8-8002-474b-805b-c877e1d4e868'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:20.999491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] The token cache is being invalidate because project 140a5d8a1d264d8bad17d352591fa6a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:20.999771 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:21.000382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-13cfd2b8-8002-474b-805b-c877e1d4e868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:21.001870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1295/2578] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:20 2022] DELETE /identity/v3/projects/140a5d8a1d264d8bad17d352591fa6a2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:21.007418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f68ed-1576-4485-9d45-feaf28e24935 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.007704 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f68ed-1576-4485-9d45-feaf28e24935 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.007979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f68ed-1576-4485-9d45-feaf28e24935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.028003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-0c8f68ed-1576-4485-9d45-feaf28e24935 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:21.029202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0c8f68ed-1576-4485-9d45-feaf28e24935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:21.087057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1284/2579] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:21.094741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e967eb-3102-4cc2-9456-1be226085ab6 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.095018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e967eb-3102-4cc2-9456-1be226085ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.095284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e967eb-3102-4cc2-9456-1be226085ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.114759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c4e967eb-3102-4cc2-9456-1be226085ab6 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:21.115876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c4e967eb-3102-4cc2-9456-1be226085ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:21.123402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1296/2580] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:21.127490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f68ed-1576-4485-9d45-feaf28e24935 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:21.137666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f68ed-1576-4485-9d45-feaf28e24935 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:21.142249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cfc95-1d64-409e-a471-504303f72abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:21.142843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cfc95-1d64-409e-a471-504303f72abc admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.143121 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cfc95-1d64-409e-a471-504303f72abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.143405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cfc95-1d64-409e-a471-504303f72abc admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.144035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cfc95-1d64-409e-a471-504303f72abc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:21.144338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cfc95-1d64-409e-a471-504303f72abc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:21.145973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cfc95-1d64-409e-a471-504303f72abc admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:21.158009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1285/2581] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:29:21.162247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e967eb-3102-4cc2-9456-1be226085ab6 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:21.171264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e967eb-3102-4cc2-9456-1be226085ab6 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:21.176304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ab690-8457-4de2-be6d-9fc7b62822d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:21.176909 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ab690-8457-4de2-be6d-9fc7b62822d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.177178 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ab690-8457-4de2-be6d-9fc7b62822d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.177443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ab690-8457-4de2-be6d-9fc7b62822d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.178086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ab690-8457-4de2-be6d-9fc7b62822d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:21.179956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ab690-8457-4de2-be6d-9fc7b62822d3 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:21.207657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1297/2582] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:21 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:21.211628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cfc95-1d64-409e-a471-504303f72abc admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:21.221624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cfc95-1d64-409e-a471-504303f72abc admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:21.277064 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dde4b-f924-4d3a-b574-3d13da6dd5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:21.277690 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dde4b-f924-4d3a-b574-3d13da6dd5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.277970 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dde4b-f924-4d3a-b574-3d13da6dd5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.278242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dde4b-f924-4d3a-b574-3d13da6dd5f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.278859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dde4b-f924-4d3a-b574-3d13da6dd5f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:21.280770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dde4b-f924-4d3a-b574-3d13da6dd5f7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:21.302151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1286/2583] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:21 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:21.306236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ab690-8457-4de2-be6d-9fc7b62822d3 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:21.316816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ab690-8457-4de2-be6d-9fc7b62822d3 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:21.321625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda3784-a652-4dcd-bd31-b418754a7557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:21.322208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda3784-a652-4dcd-bd31-b418754a7557 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.322480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda3784-a652-4dcd-bd31-b418754a7557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.322764 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda3784-a652-4dcd-bd31-b418754a7557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.323401 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda3784-a652-4dcd-bd31-b418754a7557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:21.324961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda3784-a652-4dcd-bd31-b418754a7557 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:21.334554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1298/2584] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:21.338341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dde4b-f924-4d3a-b574-3d13da6dd5f7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:21.348303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dde4b-f924-4d3a-b574-3d13da6dd5f7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:21.353073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438a5ab-7b01-4921-b65a-5b06122222f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:21.353702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438a5ab-7b01-4921-b65a-5b06122222f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.353980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438a5ab-7b01-4921-b65a-5b06122222f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.354249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438a5ab-7b01-4921-b65a-5b06122222f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.354884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438a5ab-7b01-4921-b65a-5b06122222f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:21.356461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438a5ab-7b01-4921-b65a-5b06122222f4 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:21.367634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1287/2585] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:21.371544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda3784-a652-4dcd-bd31-b418754a7557 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:21.381809 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda3784-a652-4dcd-bd31-b418754a7557 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:21.386282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80054b76-577a-4407-bfd9-28974c37b8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:21.386911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80054b76-577a-4407-bfd9-28974c37b8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.387184 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80054b76-577a-4407-bfd9-28974c37b8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.387452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80054b76-577a-4407-bfd9-28974c37b8f7 admin admin] PATH_INFO: `/v3/projects/22d8db7591d54cbc87780390a27a91cd/users/35de8eb330924aa69c1c27402ad3a351/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.401366 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80054b76-577a-4407-bfd9-28974c37b8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d8db7591d54cbc87780390a27a91cd, user_id=35de8eb330924aa69c1c27402ad3a351, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:21.404000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80054b76-577a-4407-bfd9-28974c37b8f7 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:21.417059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1299/2586] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:21 2022] PUT /identity/v3/projects/22d8db7591d54cbc87780390a27a91cd/users/35de8eb330924aa69c1c27402ad3a351/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:21.427709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90216e-2b1d-4904-8648-b9d3f7e7e0fd None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.428072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90216e-2b1d-4904-8648-b9d3f7e7e0fd None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.428350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90216e-2b1d-4904-8648-b9d3f7e7e0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.456886 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8c90216e-2b1d-4904-8648-b9d3f7e7e0fd None None] MFA Rules not processed for user `35de8eb330924aa69c1c27402ad3a351`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:21.458116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8c90216e-2b1d-4904-8648-b9d3f7e7e0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:21.477453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:21.521899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1288/2587] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:21.532703 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80054b76-577a-4407-bfd9-28974c37b8f7 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:21.534386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-80054b76-577a-4407-bfd9-28974c37b8f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:21.590483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80054b76-577a-4407-bfd9-28974c37b8f7 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:21.594171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b08768b-80d5-4bd6-8b5f-6648c105ddeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:21.594769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08768b-80d5-4bd6-8b5f-6648c105ddeb service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:21.595044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08768b-80d5-4bd6-8b5f-6648c105ddeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:21.595313 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08768b-80d5-4bd6-8b5f-6648c105ddeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:21.604232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08768b-80d5-4bd6-8b5f-6648c105ddeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:21.605680 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b08768b-80d5-4bd6-8b5f-6648c105ddeb service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:21.615208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1300/2588] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:28.586153 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90216e-2b1d-4904-8648-b9d3f7e7e0fd None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:28.587564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8c90216e-2b1d-4904-8648-b9d3f7e7e0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:28.651050 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90216e-2b1d-4904-8648-b9d3f7e7e0fd None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:28.655839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:28.656512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:28.656849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:28.657151 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] PATH_INFO: `/v3/users/fe1ed657cc3f4dcb80f1559adfee5c7f` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:28.658405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe1ed657cc3f4dcb80f1559adfee5c7f)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:28.661229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:28.705929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe1ed657cc3f4dcb80f1559adfee5c7f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:28.706564 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe1ed657cc3f4dcb80f1559adfee5c7f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:28.713883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe1ed657cc3f4dcb80f1559adfee5c7f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:28.720795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe1ed657cc3f4dcb80f1559adfee5c7f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:28.721411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe1ed657cc3f4dcb80f1559adfee5c7f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:28.738772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1289/2589] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:28 2022] DELETE /identity/v3/users/fe1ed657cc3f4dcb80f1559adfee5c7f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:28.743392 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb1cf7-50b6-4ead-8241-25f21bc02e3b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:28.743898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb1cf7-50b6-4ead-8241-25f21bc02e3b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:28.744010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb1cf7-50b6-4ead-8241-25f21bc02e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:28.762592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-9afb1cf7-50b6-4ead-8241-25f21bc02e3b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:28.763769 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9afb1cf7-50b6-4ead-8241-25f21bc02e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:28.820302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1301/2590] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:28.829939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:28.831396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:28.877697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:28.890333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aef729-7603-4737-bc33-f5900c4a0994 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:28.893883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ece91e-e2c1-458c-8ae2-6879324175e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:28.894523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ece91e-e2c1-458c-8ae2-6879324175e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:28.894798 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ece91e-e2c1-458c-8ae2-6879324175e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:28.895572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ece91e-e2c1-458c-8ae2-6879324175e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:28.905143 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ece91e-e2c1-458c-8ae2-6879324175e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:28.907026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ece91e-e2c1-458c-8ae2-6879324175e4 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:28.917499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1290/2591] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:29.189644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb1cf7-50b6-4ead-8241-25f21bc02e3b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:29.191255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9afb1cf7-50b6-4ead-8241-25f21bc02e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:29.210650 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb1cf7-50b6-4ead-8241-25f21bc02e3b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:29.215501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:29.216073 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:29.216373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:29.216805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] PATH_INFO: `/v3/projects/64da4b58571040509d92ef27a0b3223f` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:29.218082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64da4b58571040509d92ef27a0b3223f)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:29.220172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:29.222031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:29.271697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64da4b58571040509d92ef27a0b3223f', 'request_id': 'req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.279010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64da4b58571040509d92ef27a0b3223f', 'request_id': 'req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.285230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64da4b58571040509d92ef27a0b3223f', 'request_id': 'req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.295297 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] The token cache is being invalidate because project 64da4b58571040509d92ef27a0b3223f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:29.295580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.296206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.297678 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1302/2592] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:29 2022] DELETE /identity/v3/projects/64da4b58571040509d92ef27a0b3223f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:29.301643 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ece91e-e2c1-458c-8ae2-6879324175e4 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:29.311267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ece91e-e2c1-458c-8ae2-6879324175e4 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:29.368368 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:29.368994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:29.369272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:29.369573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] PATH_INFO: `/v3/users/8b70da2510f64a50bd6ea94a626d78a3` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:29.370673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b70da2510f64a50bd6ea94a626d78a3)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:29.372591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:29.419060 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b70da2510f64a50bd6ea94a626d78a3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.419707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b70da2510f64a50bd6ea94a626d78a3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.427214 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b70da2510f64a50bd6ea94a626d78a3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.434727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b70da2510f64a50bd6ea94a626d78a3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.435510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-28759021-880e-497d-a6c7-9ee9a10a12c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b70da2510f64a50bd6ea94a626d78a3'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.453191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1291/2593] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:29 2022] DELETE /identity/v3/users/8b70da2510f64a50bd6ea94a626d78a3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:29.730841 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:29.732326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:29.747444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4c7cd-4efc-4595-b9ab-ff7bbb2b30f8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:29.806064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:29.806688 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:29.806967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:29.807239 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] PATH_INFO: `/v3/projects/48da174e1b1247519cd78caa1de305d2` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:29.808338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48da174e1b1247519cd78caa1de305d2)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:29.810279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:29.855348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48da174e1b1247519cd78caa1de305d2', 'request_id': 'req-0c4a66b5-2005-479d-901e-2544c04dfa81'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.862591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48da174e1b1247519cd78caa1de305d2', 'request_id': 'req-0c4a66b5-2005-479d-901e-2544c04dfa81'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.869855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48da174e1b1247519cd78caa1de305d2', 'request_id': 'req-0c4a66b5-2005-479d-901e-2544c04dfa81'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.880504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] The token cache is being invalidate because project 48da174e1b1247519cd78caa1de305d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:29.880794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.881477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-0c4a66b5-2005-479d-901e-2544c04dfa81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:29.883067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1303/2594] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:29 2022] DELETE /identity/v3/projects/48da174e1b1247519cd78caa1de305d2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:29.889008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da145264-af02-49b4-84bf-b81b58a18e81 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:29.889423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da145264-af02-49b4-84bf-b81b58a18e81 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:29.889591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da145264-af02-49b4-84bf-b81b58a18e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:29.910334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-da145264-af02-49b4-84bf-b81b58a18e81 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:29.912065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-da145264-af02-49b4-84bf-b81b58a18e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:29.969430 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1292/2595] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:29.977969 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b9ea7-cd49-4717-85ec-c3fbff4523bd None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:29.978263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b9ea7-cd49-4717-85ec-c3fbff4523bd None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:29.978537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b9ea7-cd49-4717-85ec-c3fbff4523bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:29.998589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e54b9ea7-cd49-4717-85ec-c3fbff4523bd None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:29.999782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e54b9ea7-cd49-4717-85ec-c3fbff4523bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.007436 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1304/2596] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:30.011544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da145264-af02-49b4-84bf-b81b58a18e81 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.017580 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:30.021651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da145264-af02-49b4-84bf-b81b58a18e81 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.027593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.028226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.028508 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.028828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.029474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:30.029977 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.032134 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.047402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1293/2597] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:29:30.048541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b9ea7-cd49-4717-85ec-c3fbff4523bd None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.058723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b9ea7-cd49-4717-85ec-c3fbff4523bd None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.063335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.064988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.066248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc225d-17e0-4fa7-aad8-2e1b93f2fbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.066856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc225d-17e0-4fa7-aad8-2e1b93f2fbc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.067134 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc225d-17e0-4fa7-aad8-2e1b93f2fbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.067670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc225d-17e0-4fa7-aad8-2e1b93f2fbc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.068325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc225d-17e0-4fa7-aad8-2e1b93f2fbc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.070283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc225d-17e0-4fa7-aad8-2e1b93f2fbc5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.077837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417697-aafd-4d35-a101-cdbbf2742745 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.083052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.083545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:30.083675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.083926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.084217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] PATH_INFO: `/v3/users/35de8eb330924aa69c1c27402ad3a351` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.085581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=35de8eb330924aa69c1c27402ad3a351)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.087524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.105351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1305/2598] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:30 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:30.106344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc225d-17e0-4fa7-aad8-2e1b93f2fbc5 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.135566 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35de8eb330924aa69c1c27402ad3a351'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.136212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35de8eb330924aa69c1c27402ad3a351'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.143435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35de8eb330924aa69c1c27402ad3a351'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.152001 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35de8eb330924aa69c1c27402ad3a351'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.152757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb9cb918-7380-4baa-b7e4-13ad016f8c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35de8eb330924aa69c1c27402ad3a351'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.171386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc225d-17e0-4fa7-aad8-2e1b93f2fbc5 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.172423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1294/2599] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:30 2022] DELETE /identity/v3/users/35de8eb330924aa69c1c27402ad3a351 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:30.176757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159a13d-95bc-41eb-83c4-8a7204abf158 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.177035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159a13d-95bc-41eb-83c4-8a7204abf158 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.177305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159a13d-95bc-41eb-83c4-8a7204abf158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.195786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8159a13d-95bc-41eb-83c4-8a7204abf158 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:30.196975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8159a13d-95bc-41eb-83c4-8a7204abf158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.228248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29d9da-d384-4c29-81c3-c1e03bca5e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.228787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29d9da-d384-4c29-81c3-c1e03bca5e32 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.229069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29d9da-d384-4c29-81c3-c1e03bca5e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.229347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29d9da-d384-4c29-81c3-c1e03bca5e32 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.230083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29d9da-d384-4c29-81c3-c1e03bca5e32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.232084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29d9da-d384-4c29-81c3-c1e03bca5e32 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.253554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1306/2600] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:30 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:30.257708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29d9da-d384-4c29-81c3-c1e03bca5e32 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.257940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1295/2601] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:30.262346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9c29d9da-d384-4c29-81c3-c1e03bca5e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.267618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159a13d-95bc-41eb-83c4-8a7204abf158 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.268951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8159a13d-95bc-41eb-83c4-8a7204abf158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.274656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29d9da-d384-4c29-81c3-c1e03bca5e32 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.279274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b65572-15d7-47dc-8a48-74c2f2e4c825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.280250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b65572-15d7-47dc-8a48-74c2f2e4c825 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.280523 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b65572-15d7-47dc-8a48-74c2f2e4c825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.280792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b65572-15d7-47dc-8a48-74c2f2e4c825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.281428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b65572-15d7-47dc-8a48-74c2f2e4c825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.283026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b65572-15d7-47dc-8a48-74c2f2e4c825 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.293697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1307/2602] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:30.298029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b65572-15d7-47dc-8a48-74c2f2e4c825 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.307658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b65572-15d7-47dc-8a48-74c2f2e4c825 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.312265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398bfea-f4b7-4213-9d4c-df096e262621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.312836 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398bfea-f4b7-4213-9d4c-df096e262621 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.313111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398bfea-f4b7-4213-9d4c-df096e262621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.313384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398bfea-f4b7-4213-9d4c-df096e262621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.314039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398bfea-f4b7-4213-9d4c-df096e262621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.315600 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398bfea-f4b7-4213-9d4c-df096e262621 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.326442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1308/2603] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:30.327176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159a13d-95bc-41eb-83c4-8a7204abf158 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.330552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398bfea-f4b7-4213-9d4c-df096e262621 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.330904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05c6e8-7139-4ae9-89a7-91c15760275a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.331526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05c6e8-7139-4ae9-89a7-91c15760275a service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.331933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05c6e8-7139-4ae9-89a7-91c15760275a service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.332213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05c6e8-7139-4ae9-89a7-91c15760275a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.340619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398bfea-f4b7-4213-9d4c-df096e262621 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.342131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05c6e8-7139-4ae9-89a7-91c15760275a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.343560 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05c6e8-7139-4ae9-89a7-91c15760275a service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.345414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f775c-97dd-425c-b548-901665df90db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.346086 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f775c-97dd-425c-b548-901665df90db admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.346361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f775c-97dd-425c-b548-901665df90db admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.346630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f775c-97dd-425c-b548-901665df90db admin admin] PATH_INFO: `/v3/projects/af100c985a6449dea5616094cdd5b4be/users/6dbb0550035d4b519d91f635d44d66b9/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.353752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1296/2604] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:29:30.365944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f775c-97dd-425c-b548-901665df90db admin admin] RBAC: Authorizing `identity:create_grant(project_id=af100c985a6449dea5616094cdd5b4be, user_id=6dbb0550035d4b519d91f635d44d66b9, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.368576 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f775c-97dd-425c-b548-901665df90db admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.381883 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1309/2605] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:30 2022] PUT /identity/v3/projects/af100c985a6449dea5616094cdd5b4be/users/6dbb0550035d4b519d91f635d44d66b9/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:30.393244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90472902-5f9d-44ab-97b0-ccb166e80e28 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.393441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90472902-5f9d-44ab-97b0-ccb166e80e28 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.393736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90472902-5f9d-44ab-97b0-ccb166e80e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.410243 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:30.429785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-90472902-5f9d-44ab-97b0-ccb166e80e28 None None] MFA Rules not processed for user `6dbb0550035d4b519d91f635d44d66b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:30.431164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-90472902-5f9d-44ab-97b0-ccb166e80e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.502903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1297/2606] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:30.507656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f775c-97dd-425c-b548-901665df90db admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.552284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:30.571310 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f775c-97dd-425c-b548-901665df90db admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.576034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b21c49-4ed9-48d1-8936-fc153ab0bc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.576639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b21c49-4ed9-48d1-8936-fc153ab0bc8a admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.576911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b21c49-4ed9-48d1-8936-fc153ab0bc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.577241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b21c49-4ed9-48d1-8936-fc153ab0bc8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.577905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b21c49-4ed9-48d1-8936-fc153ab0bc8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.579795 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b21c49-4ed9-48d1-8936-fc153ab0bc8a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.607358 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1310/2607] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:30 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:30.611995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90472902-5f9d-44ab-97b0-ccb166e80e28 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.623349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90472902-5f9d-44ab-97b0-ccb166e80e28 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.628402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b21c49-4ed9-48d1-8936-fc153ab0bc8a admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.632618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-82b21c49-4ed9-48d1-8936-fc153ab0bc8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.693863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05b302-4ee5-4a5e-b931-8101d3999f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.694477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b302-4ee5-4a5e-b931-8101d3999f46 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.694757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b302-4ee5-4a5e-b931-8101d3999f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.695033 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b302-4ee5-4a5e-b931-8101d3999f46 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.695666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05b302-4ee5-4a5e-b931-8101d3999f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.697593 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05b302-4ee5-4a5e-b931-8101d3999f46 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.709724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b21c49-4ed9-48d1-8936-fc153ab0bc8a admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.715021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.715657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.715937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.716205 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] PATH_INFO: `/v3/projects/22d8db7591d54cbc87780390a27a91cd` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.717371 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d8db7591d54cbc87780390a27a91cd)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.719316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.719663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1298/2608] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:30 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:30.723790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05b302-4ee5-4a5e-b931-8101d3999f46 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.733168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05b302-4ee5-4a5e-b931-8101d3999f46 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.737802 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b61807-5cda-4cf1-b606-6c4c3d5af80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.738374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b61807-5cda-4cf1-b606-6c4c3d5af80f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.738696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b61807-5cda-4cf1-b606-6c4c3d5af80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.738963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b61807-5cda-4cf1-b606-6c4c3d5af80f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.739616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b61807-5cda-4cf1-b606-6c4c3d5af80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.741205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b61807-5cda-4cf1-b606-6c4c3d5af80f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.756510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1299/2609] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:30.757537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b61807-5cda-4cf1-b606-6c4c3d5af80f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.767093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b61807-5cda-4cf1-b606-6c4c3d5af80f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.767820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d8db7591d54cbc87780390a27a91cd', 'request_id': 'req-bd1765ed-34d2-49ba-a503-5ec752ea8809'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.776395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d8db7591d54cbc87780390a27a91cd', 'request_id': 'req-bd1765ed-34d2-49ba-a503-5ec752ea8809'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.783012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d8db7591d54cbc87780390a27a91cd', 'request_id': 'req-bd1765ed-34d2-49ba-a503-5ec752ea8809'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.793542 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] The token cache is being invalidate because project 22d8db7591d54cbc87780390a27a91cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:30.793831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.794544 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-bd1765ed-34d2-49ba-a503-5ec752ea8809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:30.796108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1311/2610] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:30 2022] DELETE /identity/v3/projects/22d8db7591d54cbc87780390a27a91cd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:30.800844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377664f2-e271-40af-b49e-a6b21ee9cced None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.801168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377664f2-e271-40af-b49e-a6b21ee9cced None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.801442 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377664f2-e271-40af-b49e-a6b21ee9cced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.822018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-377664f2-e271-40af-b49e-a6b21ee9cced None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:30.823236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-377664f2-e271-40af-b49e-a6b21ee9cced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.825997 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47525d0-963b-4966-934e-c0446a0e9c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.826659 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47525d0-963b-4966-934e-c0446a0e9c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.826940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47525d0-963b-4966-934e-c0446a0e9c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.827214 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47525d0-963b-4966-934e-c0446a0e9c4a admin admin] PATH_INFO: `/v3/projects/21a5f7e72a864e08b2fa62679ef5e807/users/016e9214220a45c58f5e6e71d9bfd464/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.831439 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1312/2611] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:30.839220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18c15-b601-4212-aca2-6ae75401e885 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.839505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18c15-b601-4212-aca2-6ae75401e885 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.839778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18c15-b601-4212-aca2-6ae75401e885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.842540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47525d0-963b-4966-934e-c0446a0e9c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a5f7e72a864e08b2fa62679ef5e807, user_id=016e9214220a45c58f5e6e71d9bfd464, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.845173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47525d0-963b-4966-934e-c0446a0e9c4a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.853227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:30.859361 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-ebf18c15-b601-4212-aca2-6ae75401e885 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:30.860397 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1300/2612] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:30 2022] PUT /identity/v3/projects/21a5f7e72a864e08b2fa62679ef5e807/users/016e9214220a45c58f5e6e71d9bfd464/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:30.863403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ebf18c15-b601-4212-aca2-6ae75401e885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.870237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47525d0-963b-4966-934e-c0446a0e9c4a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.876583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-b47525d0-963b-4966-934e-c0446a0e9c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:30.909495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:30.926903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1313/2613] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:30.931262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf18c15-b601-4212-aca2-6ae75401e885 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.940207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf18c15-b601-4212-aca2-6ae75401e885 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.945883 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47525d0-963b-4966-934e-c0446a0e9c4a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.946388 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d03a40-ddf5-4ee9-9b48-4d365a0ae97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.946749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d03a40-ddf5-4ee9-9b48-4d365a0ae97f admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.947026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d03a40-ddf5-4ee9-9b48-4d365a0ae97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.947299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d03a40-ddf5-4ee9-9b48-4d365a0ae97f admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.947935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d03a40-ddf5-4ee9-9b48-4d365a0ae97f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:30.948235 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d03a40-ddf5-4ee9-9b48-4d365a0ae97f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.949853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d03a40-ddf5-4ee9-9b48-4d365a0ae97f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.950683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f1a4c-0522-4f63-8c86-26e125e3c5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.951282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1a4c-0522-4f63-8c86-26e125e3c5dd admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.951555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1a4c-0522-4f63-8c86-26e125e3c5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.951819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1a4c-0522-4f63-8c86-26e125e3c5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.952445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f1a4c-0522-4f63-8c86-26e125e3c5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.954084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f1a4c-0522-4f63-8c86-26e125e3c5dd admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.961689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1314/2614] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:29:30.964150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1301/2615] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:30.965949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d03a40-ddf5-4ee9-9b48-4d365a0ae97f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.968021 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f1a4c-0522-4f63-8c86-26e125e3c5dd admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:30.975770 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d03a40-ddf5-4ee9-9b48-4d365a0ae97f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.977595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f1a4c-0522-4f63-8c86-26e125e3c5dd admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:30.980655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c49e6-700d-46b0-9966-bdae9733ce6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.981291 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c49e6-700d-46b0-9966-bdae9733ce6e admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.981604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c49e6-700d-46b0-9966-bdae9733ce6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.981881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c49e6-700d-46b0-9966-bdae9733ce6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.982266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1d143-33fa-4666-b19b-8f8fb9bd44bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:30.982506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c49e6-700d-46b0-9966-bdae9733ce6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.982890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1d143-33fa-4666-b19b-8f8fb9bd44bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:30.983165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1d143-33fa-4666-b19b-8f8fb9bd44bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:30.983436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1d143-33fa-4666-b19b-8f8fb9bd44bb admin admin] PATH_INFO: `/v3/projects/21a5f7e72a864e08b2fa62679ef5e807/users/016e9214220a45c58f5e6e71d9bfd464/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:30.984518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c49e6-700d-46b0-9966-bdae9733ce6e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.985461 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1d143-33fa-4666-b19b-8f8fb9bd44bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a5f7e72a864e08b2fa62679ef5e807, user_id=016e9214220a45c58f5e6e71d9bfd464, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:30.988211 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1d143-33fa-4666-b19b-8f8fb9bd44bb admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:30.998905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1302/2616] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:30 2022] PUT /identity/v3/projects/21a5f7e72a864e08b2fa62679ef5e807/users/016e9214220a45c58f5e6e71d9bfd464/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:31.011342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0884c39-cc9a-48fa-8f81-ee26f05b3c92 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:31.011565 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0884c39-cc9a-48fa-8f81-ee26f05b3c92 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:31.011838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0884c39-cc9a-48fa-8f81-ee26f05b3c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:31.018917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1315/2617] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:30 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:31.023213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c49e6-700d-46b0-9966-bdae9733ce6e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:31.040387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e0884c39-cc9a-48fa-8f81-ee26f05b3c92 None None] MFA Rules not processed for user `016e9214220a45c58f5e6e71d9bfd464`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:31.042053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e0884c39-cc9a-48fa-8f81-ee26f05b3c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:31.085870 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c49e6-700d-46b0-9966-bdae9733ce6e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:31.090422 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f926c7-7e31-4665-ba8d-b47356971871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:31.091014 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f926c7-7e31-4665-ba8d-b47356971871 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:31.091287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f926c7-7e31-4665-ba8d-b47356971871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:31.091551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f926c7-7e31-4665-ba8d-b47356971871 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:31.092166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f926c7-7e31-4665-ba8d-b47356971871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:31.094096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f926c7-7e31-4665-ba8d-b47356971871 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:31.111360 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1303/2618] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:31.115411 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1316/2619] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:31 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:31.119500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0884c39-cc9a-48fa-8f81-ee26f05b3c92 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:31.123417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f926c7-7e31-4665-ba8d-b47356971871 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:31.125034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-f3f926c7-7e31-4665-ba8d-b47356971871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:31.130756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0884c39-cc9a-48fa-8f81-ee26f05b3c92 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:31.135372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc721a9c-1d1d-4067-a795-5efdaf5876e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:31.135961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721a9c-1d1d-4067-a795-5efdaf5876e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:31.136245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721a9c-1d1d-4067-a795-5efdaf5876e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:31.136541 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721a9c-1d1d-4067-a795-5efdaf5876e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:31.137184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc721a9c-1d1d-4067-a795-5efdaf5876e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:31.138827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc721a9c-1d1d-4067-a795-5efdaf5876e6 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:31.148650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1304/2620] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:31.152557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc721a9c-1d1d-4067-a795-5efdaf5876e6 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:31.163073 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc721a9c-1d1d-4067-a795-5efdaf5876e6 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:31.167714 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421ea0f-e0e7-40db-bde0-fbd86adfa446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:31.168299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421ea0f-e0e7-40db-bde0-fbd86adfa446 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:31.168582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421ea0f-e0e7-40db-bde0-fbd86adfa446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:31.168849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421ea0f-e0e7-40db-bde0-fbd86adfa446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:31.169470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421ea0f-e0e7-40db-bde0-fbd86adfa446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:31.171094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421ea0f-e0e7-40db-bde0-fbd86adfa446 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:31.184327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1305/2621] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:31.184510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f926c7-7e31-4665-ba8d-b47356971871 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:31.185305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421ea0f-e0e7-40db-bde0-fbd86adfa446 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:31.186470 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a3bac-3e58-47e5-a2d5-12df71efacfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:31.187068 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a3bac-3e58-47e5-a2d5-12df71efacfb service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:31.187351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a3bac-3e58-47e5-a2d5-12df71efacfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:31.187619 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a3bac-3e58-47e5-a2d5-12df71efacfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:31.195657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421ea0f-e0e7-40db-bde0-fbd86adfa446 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:31.197465 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a3bac-3e58-47e5-a2d5-12df71efacfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:31.198928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a3bac-3e58-47e5-a2d5-12df71efacfb service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:31.200226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0df2ad-f3df-4795-8547-09acc3eda4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:31.200844 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df2ad-f3df-4795-8547-09acc3eda4d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:31.201120 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df2ad-f3df-4795-8547-09acc3eda4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:31.201385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df2ad-f3df-4795-8547-09acc3eda4d7 admin admin] PATH_INFO: `/v3/projects/c3500e7a8f1e4e3a8cd4e4e8eb5af43a/users/55f2ecd1a24c49cfb2a7e8b987177b30/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:31.208587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1317/2622] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:29:31.215840 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0df2ad-f3df-4795-8547-09acc3eda4d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3500e7a8f1e4e3a8cd4e4e8eb5af43a, user_id=55f2ecd1a24c49cfb2a7e8b987177b30, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:31.218507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0df2ad-f3df-4795-8547-09acc3eda4d7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:31.233038 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1306/2623] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:31 2022] PUT /identity/v3/projects/c3500e7a8f1e4e3a8cd4e4e8eb5af43a/users/55f2ecd1a24c49cfb2a7e8b987177b30/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:31.238206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766656f-af7a-497d-9dd5-b24a545afe03 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:31.239225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766656f-af7a-497d-9dd5-b24a545afe03 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:31.239716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766656f-af7a-497d-9dd5-b24a545afe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:31.258818 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1766656f-af7a-497d-9dd5-b24a545afe03 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:31.260016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1766656f-af7a-497d-9dd5-b24a545afe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:31.292502 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0df2ad-f3df-4795-8547-09acc3eda4d7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:31.305109 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:31.319862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:31.322632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1318/2624] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:31.332620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766656f-af7a-497d-9dd5-b24a545afe03 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:31.334036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1766656f-af7a-497d-9dd5-b24a545afe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:31.359535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0df2ad-f3df-4795-8547-09acc3eda4d7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:31.363237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f395909-238e-4f14-b05c-b6839cf1dd2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:31.364451 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f395909-238e-4f14-b05c-b6839cf1dd2a service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:31.364735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f395909-238e-4f14-b05c-b6839cf1dd2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:31.365084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f395909-238e-4f14-b05c-b6839cf1dd2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:31.366896 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1f395909-238e-4f14-b05c-b6839cf1dd2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:31.390312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766656f-af7a-497d-9dd5-b24a545afe03 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:31.394000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae9219-da7a-448d-957a-f506d9ae776c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:31.394598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae9219-da7a-448d-957a-f506d9ae776c service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:31.394906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae9219-da7a-448d-957a-f506d9ae776c service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:31.395174 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae9219-da7a-448d-957a-f506d9ae776c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:31.404282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae9219-da7a-448d-957a-f506d9ae776c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:31.405747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae9219-da7a-448d-957a-f506d9ae776c service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:31.415674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1319/2625] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:29:31.428521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f395909-238e-4f14-b05c-b6839cf1dd2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:31.430049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f395909-238e-4f14-b05c-b6839cf1dd2a service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:31.439066 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1307/2626] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:29:32.068277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a5134-acbc-4577-bda9-afcddf4ad2c9 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:32.068791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a5134-acbc-4577-bda9-afcddf4ad2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:32.068966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a5134-acbc-4577-bda9-afcddf4ad2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:32.098660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8c0a5134-acbc-4577-bda9-afcddf4ad2c9 None None] MFA Rules not processed for user `55f2ecd1a24c49cfb2a7e8b987177b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:32.099874 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8c0a5134-acbc-4577-bda9-afcddf4ad2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:32.172430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1320/2627] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:32 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:32.176902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f395909-238e-4f14-b05c-b6839cf1dd2a service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:32.187546 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f395909-238e-4f14-b05c-b6839cf1dd2a service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:32.192699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb146024-23ec-4385-b521-b236fe0b9958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:32.193286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb146024-23ec-4385-b521-b236fe0b9958 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:32.193589 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb146024-23ec-4385-b521-b236fe0b9958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:32.193984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb146024-23ec-4385-b521-b236fe0b9958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:32.194621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb146024-23ec-4385-b521-b236fe0b9958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:32.196513 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb146024-23ec-4385-b521-b236fe0b9958 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:32.226796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1308/2628] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:32 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:32.228245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a5134-acbc-4577-bda9-afcddf4ad2c9 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:32.238855 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a5134-acbc-4577-bda9-afcddf4ad2c9 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:32.294636 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49036aac-665d-4177-9725-0ce083d42160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:32.295237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49036aac-665d-4177-9725-0ce083d42160 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:32.295512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49036aac-665d-4177-9725-0ce083d42160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:32.295778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49036aac-665d-4177-9725-0ce083d42160 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:32.296403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49036aac-665d-4177-9725-0ce083d42160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:32.298330 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49036aac-665d-4177-9725-0ce083d42160 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:32.319440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1321/2629] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:32 2022] POST /identity/v3/users => generated 309 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:32.323388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb146024-23ec-4385-b521-b236fe0b9958 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:32.334268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb146024-23ec-4385-b521-b236fe0b9958 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:32.338758 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a493c0-42dc-4b00-b2e6-08264e5a6028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:32.339351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a493c0-42dc-4b00-b2e6-08264e5a6028 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:32.339628 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a493c0-42dc-4b00-b2e6-08264e5a6028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:32.339897 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a493c0-42dc-4b00-b2e6-08264e5a6028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:32.340531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a493c0-42dc-4b00-b2e6-08264e5a6028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:32.342146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a493c0-42dc-4b00-b2e6-08264e5a6028 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:32.355728 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1309/2630] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:32.360108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49036aac-665d-4177-9725-0ce083d42160 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:32.370482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49036aac-665d-4177-9725-0ce083d42160 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:32.375013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdd33c-c592-4974-a0e0-1e566886a173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:32.375652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdd33c-c592-4974-a0e0-1e566886a173 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:32.375927 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdd33c-c592-4974-a0e0-1e566886a173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:32.376199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdd33c-c592-4974-a0e0-1e566886a173 admin admin] PATH_INFO: `/v3/projects/f376e4eb14c545ba9860f6ecfb9a880c/users/41ee6acfdc8241ffbd0683c5fd879a61/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:32.385111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:32.392403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdd33c-c592-4974-a0e0-1e566886a173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f376e4eb14c545ba9860f6ecfb9a880c, user_id=41ee6acfdc8241ffbd0683c5fd879a61, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:32.395100 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdd33c-c592-4974-a0e0-1e566886a173 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:32.407848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1322/2631] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:32 2022] PUT /identity/v3/projects/f376e4eb14c545ba9860f6ecfb9a880c/users/41ee6acfdc8241ffbd0683c5fd879a61/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:32.411647 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a493c0-42dc-4b00-b2e6-08264e5a6028 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:32.421606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a493c0-42dc-4b00-b2e6-08264e5a6028 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:32.476846 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da04ac-7435-45a2-b623-d5d6b7e23a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:32.477439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da04ac-7435-45a2-b623-d5d6b7e23a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:32.477741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da04ac-7435-45a2-b623-d5d6b7e23a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:32.478008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da04ac-7435-45a2-b623-d5d6b7e23a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:32.478631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da04ac-7435-45a2-b623-d5d6b7e23a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:32.480192 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da04ac-7435-45a2-b623-d5d6b7e23a88 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:32.490115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1310/2632] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:32.494172 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdd33c-c592-4974-a0e0-1e566886a173 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:32.504814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdd33c-c592-4974-a0e0-1e566886a173 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:32.509257 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bfeff-ddbe-405e-8954-04d00e60f81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:32.509931 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bfeff-ddbe-405e-8954-04d00e60f81b admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:32.510208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bfeff-ddbe-405e-8954-04d00e60f81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:32.510477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bfeff-ddbe-405e-8954-04d00e60f81b admin admin] PATH_INFO: `/v3/projects/f376e4eb14c545ba9860f6ecfb9a880c/users/41ee6acfdc8241ffbd0683c5fd879a61/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:32.512369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bfeff-ddbe-405e-8954-04d00e60f81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f376e4eb14c545ba9860f6ecfb9a880c, user_id=41ee6acfdc8241ffbd0683c5fd879a61, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:32.515013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bfeff-ddbe-405e-8954-04d00e60f81b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:32.524846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1323/2633] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:32 2022] PUT /identity/v3/projects/f376e4eb14c545ba9860f6ecfb9a880c/users/41ee6acfdc8241ffbd0683c5fd879a61/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:33.182929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631578bc-e1d7-4024-9745-7e145a255b4f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:33.183457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631578bc-e1d7-4024-9745-7e145a255b4f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:33.183557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631578bc-e1d7-4024-9745-7e145a255b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:33.211948 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-631578bc-e1d7-4024-9745-7e145a255b4f None None] MFA Rules not processed for user `41ee6acfdc8241ffbd0683c5fd879a61`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:33.213148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-631578bc-e1d7-4024-9745-7e145a255b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:33.244341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:33.291334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1311/2634] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:34.322093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bfeff-ddbe-405e-8954-04d00e60f81b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:34.393664 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bfeff-ddbe-405e-8954-04d00e60f81b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:34.397560 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:34.398215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:34.398518 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:34.398786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] PATH_INFO: `/v3/users/55f2ecd1a24c49cfb2a7e8b987177b30` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:34.399902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55f2ecd1a24c49cfb2a7e8b987177b30)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:34.401941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:34.451690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55f2ecd1a24c49cfb2a7e8b987177b30'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.452362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55f2ecd1a24c49cfb2a7e8b987177b30'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.464583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55f2ecd1a24c49cfb2a7e8b987177b30'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.465259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55f2ecd1a24c49cfb2a7e8b987177b30'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.472275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55f2ecd1a24c49cfb2a7e8b987177b30'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.490932 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1324/2635] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:34 2022] DELETE /identity/v3/users/55f2ecd1a24c49cfb2a7e8b987177b30 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:34.666689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631578bc-e1d7-4024-9745-7e145a255b4f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:34.678503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631578bc-e1d7-4024-9745-7e145a255b4f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:34.741340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:34.742020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:34.742899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:34.743173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] PATH_INFO: `/v3/projects/c3500e7a8f1e4e3a8cd4e4e8eb5af43a` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:34.744314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3500e7a8f1e4e3a8cd4e4e8eb5af43a)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:34.746258 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:34.798172 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3500e7a8f1e4e3a8cd4e4e8eb5af43a', 'request_id': 'req-8557d475-a88b-4727-9642-ec9d41b9d636'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.807480 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3500e7a8f1e4e3a8cd4e4e8eb5af43a', 'request_id': 'req-8557d475-a88b-4727-9642-ec9d41b9d636'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.814241 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3500e7a8f1e4e3a8cd4e4e8eb5af43a', 'request_id': 'req-8557d475-a88b-4727-9642-ec9d41b9d636'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.825240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] The token cache is being invalidate because project c3500e7a8f1e4e3a8cd4e4e8eb5af43a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:34.825710 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.826449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.827961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1312/2636] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:34 2022] DELETE /identity/v3/projects/c3500e7a8f1e4e3a8cd4e4e8eb5af43a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:34.833011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:34.836869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:34.850630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c8144-3a87-41a2-836f-b657a890dc60 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:34.903106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:34.915966 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:34.916609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:34.917267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:34.917958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] PATH_INFO: `/v3/users/41ee6acfdc8241ffbd0683c5fd879a61` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:34.919137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ee6acfdc8241ffbd0683c5fd879a61)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:34.921090 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:34.976405 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ee6acfdc8241ffbd0683c5fd879a61'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.977112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ee6acfdc8241ffbd0683c5fd879a61'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.984967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ee6acfdc8241ffbd0683c5fd879a61'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:34.985693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ee6acfdc8241ffbd0683c5fd879a61'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:35.000914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-133b1909-f81b-479e-841d-5ddcfc65eed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ee6acfdc8241ffbd0683c5fd879a61'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:35.035213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1325/2637] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:34 2022] DELETE /identity/v3/users/41ee6acfdc8241ffbd0683c5fd879a61 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:35.228658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:35.238037 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:35.260287 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557d475-a88b-4727-9642-ec9d41b9d636 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:35.323739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:35.336916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:35.337765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.338528 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.339178 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] PATH_INFO: `/v3/projects/f376e4eb14c545ba9860f6ecfb9a880c` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.340899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f376e4eb14c545ba9860f6ecfb9a880c)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:35.343359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:35.393281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f376e4eb14c545ba9860f6ecfb9a880c', 'request_id': 'req-dba59cd6-7f33-49c2-8a42-92ed9248e41f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:35.400617 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f376e4eb14c545ba9860f6ecfb9a880c', 'request_id': 'req-dba59cd6-7f33-49c2-8a42-92ed9248e41f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:35.406806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f376e4eb14c545ba9860f6ecfb9a880c', 'request_id': 'req-dba59cd6-7f33-49c2-8a42-92ed9248e41f'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:35.416851 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] The token cache is being invalidate because project f376e4eb14c545ba9860f6ecfb9a880c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:35.417131 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:35.417773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-dba59cd6-7f33-49c2-8a42-92ed9248e41f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:35.419218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1313/2638] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:35 2022] DELETE /identity/v3/projects/f376e4eb14c545ba9860f6ecfb9a880c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:35.424332 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbf5c4-d0f3-417c-8bd1-a4feca48d015 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.424815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbf5c4-d0f3-417c-8bd1-a4feca48d015 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.424911 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbf5c4-d0f3-417c-8bd1-a4feca48d015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.445903 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-33dbf5c4-d0f3-417c-8bd1-a4feca48d015 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:35.447158 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-33dbf5c4-d0f3-417c-8bd1-a4feca48d015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:35.503924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1326/2639] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:35.512337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280e2f9-a804-4f9f-8087-6a0f3ce66559 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.512622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280e2f9-a804-4f9f-8087-6a0f3ce66559 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.512895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280e2f9-a804-4f9f-8087-6a0f3ce66559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.532236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-6280e2f9-a804-4f9f-8087-6a0f3ce66559 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:35.533450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6280e2f9-a804-4f9f-8087-6a0f3ce66559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:35.541299 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1314/2640] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:35.545467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbf5c4-d0f3-417c-8bd1-a4feca48d015 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:35.555753 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbf5c4-d0f3-417c-8bd1-a4feca48d015 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:35.561000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c3953-74c8-4965-8e5c-6251d1dcf0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:35.561683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c3953-74c8-4965-8e5c-6251d1dcf0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.561943 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c3953-74c8-4965-8e5c-6251d1dcf0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.562212 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c3953-74c8-4965-8e5c-6251d1dcf0c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.562884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c3953-74c8-4965-8e5c-6251d1dcf0c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:35.563177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c3953-74c8-4965-8e5c-6251d1dcf0c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:35.564761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c3953-74c8-4965-8e5c-6251d1dcf0c4 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:35.577034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1327/2641] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:29:35.581456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280e2f9-a804-4f9f-8087-6a0f3ce66559 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:35.590624 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280e2f9-a804-4f9f-8087-6a0f3ce66559 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:35.595206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc1eea-6d3f-4893-be1d-d28cd141f92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:35.595791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc1eea-6d3f-4893-be1d-d28cd141f92b admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.596063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc1eea-6d3f-4893-be1d-d28cd141f92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.596324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc1eea-6d3f-4893-be1d-d28cd141f92b admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.596935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc1eea-6d3f-4893-be1d-d28cd141f92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:35.598872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc1eea-6d3f-4893-be1d-d28cd141f92b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:35.625318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1315/2642] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:35 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:35.630301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c3953-74c8-4965-8e5c-6251d1dcf0c4 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:35.639928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c3953-74c8-4965-8e5c-6251d1dcf0c4 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:35.698429 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b8c2b-1e61-459c-8338-ec06a1e4e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:35.698997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b8c2b-1e61-459c-8338-ec06a1e4e55b admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.699270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b8c2b-1e61-459c-8338-ec06a1e4e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.699534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b8c2b-1e61-459c-8338-ec06a1e4e55b admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.700157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b8c2b-1e61-459c-8338-ec06a1e4e55b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:35.702093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b8c2b-1e61-459c-8338-ec06a1e4e55b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:35.709925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:35.724626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1328/2643] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:35 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:35.728926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc1eea-6d3f-4893-be1d-d28cd141f92b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:35.739838 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc1eea-6d3f-4893-be1d-d28cd141f92b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:35.744359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57643d-5b22-4be7-8296-4aa3123db119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:35.744951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57643d-5b22-4be7-8296-4aa3123db119 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.745226 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57643d-5b22-4be7-8296-4aa3123db119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.745493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57643d-5b22-4be7-8296-4aa3123db119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.746157 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57643d-5b22-4be7-8296-4aa3123db119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:35.747733 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57643d-5b22-4be7-8296-4aa3123db119 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:35.757866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1316/2644] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:35.761796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b8c2b-1e61-459c-8338-ec06a1e4e55b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:35.771761 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b8c2b-1e61-459c-8338-ec06a1e4e55b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:35.776316 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a1533-4369-4173-a306-89f7024fc822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:35.776907 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a1533-4369-4173-a306-89f7024fc822 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.777181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a1533-4369-4173-a306-89f7024fc822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.777449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a1533-4369-4173-a306-89f7024fc822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.778112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a1533-4369-4173-a306-89f7024fc822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:35.779697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a1533-4369-4173-a306-89f7024fc822 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:35.790701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1329/2645] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:35.794791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57643d-5b22-4be7-8296-4aa3123db119 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:35.805106 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57643d-5b22-4be7-8296-4aa3123db119 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:35.809716 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf424537-a6d8-4a52-a516-06ee0c007722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:35.810350 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424537-a6d8-4a52-a516-06ee0c007722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.810623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424537-a6d8-4a52-a516-06ee0c007722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.810891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424537-a6d8-4a52-a516-06ee0c007722 admin admin] PATH_INFO: `/v3/projects/038972a67a91472f80e4a3dbdd3b094f/users/7c0a451536fb497280ac1fe06b4d603b/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.825357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf424537-a6d8-4a52-a516-06ee0c007722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=038972a67a91472f80e4a3dbdd3b094f, user_id=7c0a451536fb497280ac1fe06b4d603b, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:35.828005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf424537-a6d8-4a52-a516-06ee0c007722 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:35.840359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1317/2646] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:35 2022] PUT /identity/v3/projects/038972a67a91472f80e4a3dbdd3b094f/users/7c0a451536fb497280ac1fe06b4d603b/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:35.852042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad921cf-2f13-4191-b699-999e549e63f8 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:35.852321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad921cf-2f13-4191-b699-999e549e63f8 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:35.852633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad921cf-2f13-4191-b699-999e549e63f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:35.880796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-aad921cf-2f13-4191-b699-999e549e63f8 None None] MFA Rules not processed for user `7c0a451536fb497280ac1fe06b4d603b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:35.882027 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-aad921cf-2f13-4191-b699-999e549e63f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:35.944186 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1330/2647] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:35.948687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf424537-a6d8-4a52-a516-06ee0c007722 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.008114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf424537-a6d8-4a52-a516-06ee0c007722 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.012662 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99c627-0671-44ca-8fa3-dee316cab5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.013253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99c627-0671-44ca-8fa3-dee316cab5aa admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.013552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99c627-0671-44ca-8fa3-dee316cab5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.013832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99c627-0671-44ca-8fa3-dee316cab5aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.014458 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99c627-0671-44ca-8fa3-dee316cab5aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.016337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99c627-0671-44ca-8fa3-dee316cab5aa admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.041995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1318/2648] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:35 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:36.045989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad921cf-2f13-4191-b699-999e549e63f8 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.056919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad921cf-2f13-4191-b699-999e549e63f8 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.111194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5c219-839d-4db1-a3f9-6c6f8efbce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.111787 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5c219-839d-4db1-a3f9-6c6f8efbce70 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.112060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5c219-839d-4db1-a3f9-6c6f8efbce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.112326 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5c219-839d-4db1-a3f9-6c6f8efbce70 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.112947 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5c219-839d-4db1-a3f9-6c6f8efbce70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.114883 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5c219-839d-4db1-a3f9-6c6f8efbce70 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.136006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1331/2649] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:36 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:36.139793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99c627-0671-44ca-8fa3-dee316cab5aa admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.146991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:36.150873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99c627-0671-44ca-8fa3-dee316cab5aa admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.156889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1cae3-46c3-4898-8f18-c14396f20ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.157916 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1cae3-46c3-4898-8f18-c14396f20ced admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.158586 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1cae3-46c3-4898-8f18-c14396f20ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.158791 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1cae3-46c3-4898-8f18-c14396f20ced admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.159658 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1cae3-46c3-4898-8f18-c14396f20ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.161894 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1cae3-46c3-4898-8f18-c14396f20ced admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.173748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1319/2650] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:36.178001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5c219-839d-4db1-a3f9-6c6f8efbce70 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.187723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5c219-839d-4db1-a3f9-6c6f8efbce70 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.193070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9389c75-3c80-4446-9dc5-8e124039f9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.193732 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9389c75-3c80-4446-9dc5-8e124039f9ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.194005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9389c75-3c80-4446-9dc5-8e124039f9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.194269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9389c75-3c80-4446-9dc5-8e124039f9ab admin admin] PATH_INFO: `/v3/projects/d8dcf6ded0504d2d8dc78c5028d4e13d/users/de9c1ed132d54181998e7b5c957fc040/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.210069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9389c75-3c80-4446-9dc5-8e124039f9ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dcf6ded0504d2d8dc78c5028d4e13d, user_id=de9c1ed132d54181998e7b5c957fc040, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.214541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9389c75-3c80-4446-9dc5-8e124039f9ab admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.229377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1332/2651] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:36 2022] PUT /identity/v3/projects/d8dcf6ded0504d2d8dc78c5028d4e13d/users/de9c1ed132d54181998e7b5c957fc040/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:36.233394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1cae3-46c3-4898-8f18-c14396f20ced admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.243568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1cae3-46c3-4898-8f18-c14396f20ced admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.306049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4821c7-3365-4e63-bae5-edb5d2877a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.309905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4821c7-3365-4e63-bae5-edb5d2877a35 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.309905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4821c7-3365-4e63-bae5-edb5d2877a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.309905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4821c7-3365-4e63-bae5-edb5d2877a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.309905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4821c7-3365-4e63-bae5-edb5d2877a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.309905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4821c7-3365-4e63-bae5-edb5d2877a35 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.310840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9389c75-3c80-4446-9dc5-8e124039f9ab admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.312739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d9389c75-3c80-4446-9dc5-8e124039f9ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:36.323379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1320/2652] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:36.324487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4821c7-3365-4e63-bae5-edb5d2877a35 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.327586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9389c75-3c80-4446-9dc5-8e124039f9ab admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.333997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.334634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.334925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.335220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] PATH_INFO: `/v3/users/6dbb0550035d4b519d91f635d44d66b9` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.335913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4821c7-3365-4e63-bae5-edb5d2877a35 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.336451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dbb0550035d4b519d91f635d44d66b9)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.338516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.340740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143b56d-f9c3-4c92-8f27-02b381d550ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.341623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143b56d-f9c3-4c92-8f27-02b381d550ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.341973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143b56d-f9c3-4c92-8f27-02b381d550ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.342314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143b56d-f9c3-4c92-8f27-02b381d550ae admin admin] PATH_INFO: `/v3/projects/d8dcf6ded0504d2d8dc78c5028d4e13d/users/de9c1ed132d54181998e7b5c957fc040/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.344368 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143b56d-f9c3-4c92-8f27-02b381d550ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dcf6ded0504d2d8dc78c5028d4e13d, user_id=de9c1ed132d54181998e7b5c957fc040, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.350682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143b56d-f9c3-4c92-8f27-02b381d550ae admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.362452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1321/2653] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:36 2022] PUT /identity/v3/projects/d8dcf6ded0504d2d8dc78c5028d4e13d/users/de9c1ed132d54181998e7b5c957fc040/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:36.374344 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:INSERT=7,keystone:SELECT=88,keystone:DELETE=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:36.376605 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97516a85-1328-46c1-afbe-9d5e58899766 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.376915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97516a85-1328-46c1-afbe-9d5e58899766 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.377209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97516a85-1328-46c1-afbe-9d5e58899766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.399683 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dbb0550035d4b519d91f635d44d66b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:36.400648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dbb0550035d4b519d91f635d44d66b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:36.408127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dbb0550035d4b519d91f635d44d66b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:36.408898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dbb0550035d4b519d91f635d44d66b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:36.415015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-97516a85-1328-46c1-afbe-9d5e58899766 None None] MFA Rules not processed for user `de9c1ed132d54181998e7b5c957fc040`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:36.416382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-97516a85-1328-46c1-afbe-9d5e58899766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:36.418566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-98ce532b-c95a-4821-9d4a-5372d1a0ef1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dbb0550035d4b519d91f635d44d66b9'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:36.437466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1333/2654] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:36 2022] DELETE /identity/v3/users/6dbb0550035d4b519d91f635d44d66b9 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:36.443165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8392ed-f62c-4112-b9f5-7201d3b36d5b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.443844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8392ed-f62c-4112-b9f5-7201d3b36d5b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.444119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8392ed-f62c-4112-b9f5-7201d3b36d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.466331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5b8392ed-f62c-4112-b9f5-7201d3b36d5b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:36.467665 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5b8392ed-f62c-4112-b9f5-7201d3b36d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:36.498944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1322/2655] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:36 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:36.511135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97516a85-1328-46c1-afbe-9d5e58899766 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.512697 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-97516a85-1328-46c1-afbe-9d5e58899766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:36.528032 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1334/2656] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:36.539586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8392ed-f62c-4112-b9f5-7201d3b36d5b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.541275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5b8392ed-f62c-4112-b9f5-7201d3b36d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:36.582243 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97516a85-1328-46c1-afbe-9d5e58899766 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.599123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:36.611953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8392ed-f62c-4112-b9f5-7201d3b36d5b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.616886 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25961f97-0eab-4482-949b-a858e9bea338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.617592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25961f97-0eab-4482-949b-a858e9bea338 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.617828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25961f97-0eab-4482-949b-a858e9bea338 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.618099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25961f97-0eab-4482-949b-a858e9bea338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.628152 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25961f97-0eab-4482-949b-a858e9bea338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.629674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25961f97-0eab-4482-949b-a858e9bea338 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.641304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1335/2657] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:29:36.648273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446c12a-c050-42f1-a102-899845082fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.649450 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c12a-c050-42f1-a102-899845082fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.649768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c12a-c050-42f1-a102-899845082fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.650051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446c12a-c050-42f1-a102-899845082fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.725214 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c12a-c050-42f1-a102-899845082fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.726993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446c12a-c050-42f1-a102-899845082fe0 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.739673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1323/2658] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:36 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:29:36.795831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25961f97-0eab-4482-949b-a858e9bea338 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.806195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25961f97-0eab-4482-949b-a858e9bea338 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:36.810130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c1903-6648-490c-a6d0-c90b428667c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:36.810832 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c1903-6648-490c-a6d0-c90b428667c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:36.811165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c1903-6648-490c-a6d0-c90b428667c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:36.811497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c1903-6648-490c-a6d0-c90b428667c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:36.813328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-958c1903-6648-490c-a6d0-c90b428667c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:36.892352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:36.912835 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c1903-6648-490c-a6d0-c90b428667c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:36.914312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c1903-6648-490c-a6d0-c90b428667c9 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:36.923710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1336/2659] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:36.980674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446c12a-c050-42f1-a102-899845082fe0 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:36.982182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e446c12a-c050-42f1-a102-899845082fe0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:36.996029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446c12a-c050-42f1-a102-899845082fe0 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:37.000901 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:37.001565 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:37.001971 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:37.002248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] PATH_INFO: `/v3/projects/af100c985a6449dea5616094cdd5b4be` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:37.003365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=af100c985a6449dea5616094cdd5b4be)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:37.005416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:37.056797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af100c985a6449dea5616094cdd5b4be', 'request_id': 'req-27792f4b-50c6-43f0-8f6a-551bd879fb9a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.064229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af100c985a6449dea5616094cdd5b4be', 'request_id': 'req-27792f4b-50c6-43f0-8f6a-551bd879fb9a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.071959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af100c985a6449dea5616094cdd5b4be', 'request_id': 'req-27792f4b-50c6-43f0-8f6a-551bd879fb9a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.084797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] The token cache is being invalidate because project af100c985a6449dea5616094cdd5b4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:37.085081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.085763 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.087264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1324/2660] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:36 2022] DELETE /identity/v3/projects/af100c985a6449dea5616094cdd5b4be => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:37.091432 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c1903-6648-490c-a6d0-c90b428667c9 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:37.103080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c1903-6648-490c-a6d0-c90b428667c9 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:37.168467 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:37.169594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:37.169767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:37.170077 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] PATH_INFO: `/v3/users/016e9214220a45c58f5e6e71d9bfd464` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:37.171352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016e9214220a45c58f5e6e71d9bfd464)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:37.173462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:37.235725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016e9214220a45c58f5e6e71d9bfd464'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.237275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016e9214220a45c58f5e6e71d9bfd464'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.246524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016e9214220a45c58f5e6e71d9bfd464'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.247295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016e9214220a45c58f5e6e71d9bfd464'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.254942 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-d21a324b-ba71-4a33-91cc-b88d89083f39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016e9214220a45c58f5e6e71d9bfd464'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.274110 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1337/2661] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:37 2022] DELETE /identity/v3/users/016e9214220a45c58f5e6e71d9bfd464 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:37.541974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:37.543432 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:37.557701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27792f4b-50c6-43f0-8f6a-551bd879fb9a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:37.730369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:37.741132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:37.741749 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:37.741990 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:37.742257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] PATH_INFO: `/v3/projects/21a5f7e72a864e08b2fa62679ef5e807` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:37.743352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a5f7e72a864e08b2fa62679ef5e807)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:37.745233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:37.792411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a5f7e72a864e08b2fa62679ef5e807', 'request_id': 'req-ccb19010-adb8-4d05-b22f-f1ee20dd545a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.800411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a5f7e72a864e08b2fa62679ef5e807', 'request_id': 'req-ccb19010-adb8-4d05-b22f-f1ee20dd545a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.806940 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a5f7e72a864e08b2fa62679ef5e807', 'request_id': 'req-ccb19010-adb8-4d05-b22f-f1ee20dd545a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.817420 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] The token cache is being invalidate because project 21a5f7e72a864e08b2fa62679ef5e807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:37.817741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.818367 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ccb19010-adb8-4d05-b22f-f1ee20dd545a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:37.819859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1325/2662] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:37 2022] DELETE /identity/v3/projects/21a5f7e72a864e08b2fa62679ef5e807 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:37.824715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799e827-929f-4a4f-9c37-c32f09014562 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:37.825181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799e827-929f-4a4f-9c37-c32f09014562 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:37.825293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799e827-929f-4a4f-9c37-c32f09014562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:37.845057 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5799e827-929f-4a4f-9c37-c32f09014562 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:37.846304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5799e827-929f-4a4f-9c37-c32f09014562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:37.904812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1338/2663] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:37.913257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bcfe0-d423-4921-874f-9ee66eb40567 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:37.913557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bcfe0-d423-4921-874f-9ee66eb40567 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:37.913830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bcfe0-d423-4921-874f-9ee66eb40567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:37.932650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-507bcfe0-d423-4921-874f-9ee66eb40567 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:37.933889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-507bcfe0-d423-4921-874f-9ee66eb40567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:37.941477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1326/2664] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:37.945746 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799e827-929f-4a4f-9c37-c32f09014562 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:37.955114 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799e827-929f-4a4f-9c37-c32f09014562 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:37.960067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da535d-10d3-45b4-b397-0942a58920e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:37.960651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da535d-10d3-45b4-b397-0942a58920e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:37.960926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da535d-10d3-45b4-b397-0942a58920e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:37.961190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da535d-10d3-45b4-b397-0942a58920e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:37.961853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da535d-10d3-45b4-b397-0942a58920e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:37.962149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da535d-10d3-45b4-b397-0942a58920e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:37.963730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da535d-10d3-45b4-b397-0942a58920e4 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:37.965126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:37.979828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1339/2665] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:29:37.981798 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bcfe0-d423-4921-874f-9ee66eb40567 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:37.992592 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bcfe0-d423-4921-874f-9ee66eb40567 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:37.997024 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a51cc-012c-47a6-8da5-04346b3b6fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:37.997640 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a51cc-012c-47a6-8da5-04346b3b6fbd admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:37.997960 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a51cc-012c-47a6-8da5-04346b3b6fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:37.998220 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a51cc-012c-47a6-8da5-04346b3b6fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:37.998825 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a51cc-012c-47a6-8da5-04346b3b6fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:38.000684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a51cc-012c-47a6-8da5-04346b3b6fbd admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:38.027466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1327/2666] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:37 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:38.031679 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da535d-10d3-45b4-b397-0942a58920e4 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:38.041474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da535d-10d3-45b4-b397-0942a58920e4 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:38.096385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e366f-4886-45f2-9894-0e7acb42c201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:38.097005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e366f-4886-45f2-9894-0e7acb42c201 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:38.097278 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e366f-4886-45f2-9894-0e7acb42c201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:38.097583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e366f-4886-45f2-9894-0e7acb42c201 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:38.098243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e366f-4886-45f2-9894-0e7acb42c201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:38.100217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e366f-4886-45f2-9894-0e7acb42c201 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:38.120954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1340/2667] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:38 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:38.125018 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a51cc-012c-47a6-8da5-04346b3b6fbd admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:38.135689 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a51cc-012c-47a6-8da5-04346b3b6fbd admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:38.140233 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3857f-033a-4704-8703-30c2347bcfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:38.140821 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3857f-033a-4704-8703-30c2347bcfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:38.141097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3857f-033a-4704-8703-30c2347bcfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:38.141364 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3857f-033a-4704-8703-30c2347bcfe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:38.142022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3857f-033a-4704-8703-30c2347bcfe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:38.143629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3857f-033a-4704-8703-30c2347bcfe8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:38.153750 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1328/2668] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:38.157822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e366f-4886-45f2-9894-0e7acb42c201 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:38.167639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e366f-4886-45f2-9894-0e7acb42c201 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:38.172243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2aebd-c4c4-4f7a-b1e0-1eb9c4f12c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:38.172851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2aebd-c4c4-4f7a-b1e0-1eb9c4f12c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:38.173126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2aebd-c4c4-4f7a-b1e0-1eb9c4f12c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:38.173394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2aebd-c4c4-4f7a-b1e0-1eb9c4f12c22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:38.174064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2aebd-c4c4-4f7a-b1e0-1eb9c4f12c22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:38.175624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2aebd-c4c4-4f7a-b1e0-1eb9c4f12c22 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:38.186001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1341/2669] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:38.189930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3857f-033a-4704-8703-30c2347bcfe8 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:38.200501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3857f-033a-4704-8703-30c2347bcfe8 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:38.205100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd19d1a-7b70-41f8-a92c-d576e3abf70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:38.205764 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd19d1a-7b70-41f8-a92c-d576e3abf70e admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:38.206040 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd19d1a-7b70-41f8-a92c-d576e3abf70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:38.206308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd19d1a-7b70-41f8-a92c-d576e3abf70e admin admin] PATH_INFO: `/v3/projects/386583f809004b959b61479ae52a4bce/users/626578b1a66740df871d037e1b6339ae/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:38.220900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd19d1a-7b70-41f8-a92c-d576e3abf70e admin admin] RBAC: Authorizing `identity:create_grant(project_id=386583f809004b959b61479ae52a4bce, user_id=626578b1a66740df871d037e1b6339ae, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:38.224027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd19d1a-7b70-41f8-a92c-d576e3abf70e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:38.236333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1329/2670] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:38 2022] PUT /identity/v3/projects/386583f809004b959b61479ae52a4bce/users/626578b1a66740df871d037e1b6339ae/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:38.248398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f232b68-4d93-4408-ba34-8e45d8834c44 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:38.248680 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f232b68-4d93-4408-ba34-8e45d8834c44 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:38.248954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f232b68-4d93-4408-ba34-8e45d8834c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:38.276783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-3f232b68-4d93-4408-ba34-8e45d8834c44 None None] MFA Rules not processed for user `626578b1a66740df871d037e1b6339ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:38.278044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3f232b68-4d93-4408-ba34-8e45d8834c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:38.343260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1342/2671] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:38.355842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd19d1a-7b70-41f8-a92c-d576e3abf70e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:38.357746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6bd19d1a-7b70-41f8-a92c-d576e3abf70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:38.419906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd19d1a-7b70-41f8-a92c-d576e3abf70e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:38.423681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b2b89-71e3-4878-b57f-749342893c78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:38.424492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b2b89-71e3-4878-b57f-749342893c78 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:38.424773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b2b89-71e3-4878-b57f-749342893c78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:38.425039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b2b89-71e3-4878-b57f-749342893c78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:38.434223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b2b89-71e3-4878-b57f-749342893c78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:38.435828 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b2b89-71e3-4878-b57f-749342893c78 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:38.445113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1330/2672] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:40.245622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f232b68-4d93-4408-ba34-8e45d8834c44 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:40.247010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3f232b68-4d93-4408-ba34-8e45d8834c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:40.315280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f232b68-4d93-4408-ba34-8e45d8834c44 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:40.320081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:40.320698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:40.320977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:40.321250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] PATH_INFO: `/v3/users/7c0a451536fb497280ac1fe06b4d603b` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:40.322404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0a451536fb497280ac1fe06b4d603b)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:40.324318 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:40.326673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:40.375884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0a451536fb497280ac1fe06b4d603b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.376537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0a451536fb497280ac1fe06b4d603b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.383516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0a451536fb497280ac1fe06b4d603b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.384261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0a451536fb497280ac1fe06b4d603b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.391940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0a451536fb497280ac1fe06b4d603b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.412456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1343/2673] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:40 2022] DELETE /identity/v3/users/7c0a451536fb497280ac1fe06b4d603b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:40.416801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ceaca-303d-4829-befe-c2e0afea542b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:40.417297 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ceaca-303d-4829-befe-c2e0afea542b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:40.417384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ceaca-303d-4829-befe-c2e0afea542b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:40.437217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e21ceaca-303d-4829-befe-c2e0afea542b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:40.438459 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e21ceaca-303d-4829-befe-c2e0afea542b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:40.451967 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:40.500261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1331/2674] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:40.509956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:40.511323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:40.568562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd572f7-7851-41a3-b0fb-bb46b5450cf7 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:40.572182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812fcc0-453c-48ea-9bc4-d14a15684bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:40.572792 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812fcc0-453c-48ea-9bc4-d14a15684bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:40.573083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812fcc0-453c-48ea-9bc4-d14a15684bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:40.573382 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812fcc0-453c-48ea-9bc4-d14a15684bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:40.582867 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812fcc0-453c-48ea-9bc4-d14a15684bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:40.584404 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812fcc0-453c-48ea-9bc4-d14a15684bdf service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:40.593727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1344/2675] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:40.869111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ceaca-303d-4829-befe-c2e0afea542b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:40.872396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e21ceaca-303d-4829-befe-c2e0afea542b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:40.886797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ceaca-303d-4829-befe-c2e0afea542b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:40.891412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:40.892043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:40.892322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:40.892591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] PATH_INFO: `/v3/projects/038972a67a91472f80e4a3dbdd3b094f` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:40.893743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] RBAC: Authorizing `identity:delete_project(project_id=038972a67a91472f80e4a3dbdd3b094f)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:40.896185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:40.943034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '038972a67a91472f80e4a3dbdd3b094f', 'request_id': 'req-92807966-3992-4b4b-a41c-05fd1c00653d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.950583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '038972a67a91472f80e4a3dbdd3b094f', 'request_id': 'req-92807966-3992-4b4b-a41c-05fd1c00653d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.957642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '038972a67a91472f80e4a3dbdd3b094f', 'request_id': 'req-92807966-3992-4b4b-a41c-05fd1c00653d'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.968256 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] The token cache is being invalidate because project 038972a67a91472f80e4a3dbdd3b094f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:40.968542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.969172 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:40.970878 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1332/2676] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:40 2022] DELETE /identity/v3/projects/038972a67a91472f80e4a3dbdd3b094f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:40.975494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812fcc0-453c-48ea-9bc4-d14a15684bdf service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:40.985252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812fcc0-453c-48ea-9bc4-d14a15684bdf service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:41.040210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:41.040829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.041106 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.041375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] PATH_INFO: `/v3/users/de9c1ed132d54181998e7b5c957fc040` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.042527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9c1ed132d54181998e7b5c957fc040)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:41.044443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:41.071285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:41.091433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9c1ed132d54181998e7b5c957fc040'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.092080 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9c1ed132d54181998e7b5c957fc040'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.099440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9c1ed132d54181998e7b5c957fc040'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.100060 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9c1ed132d54181998e7b5c957fc040'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.106714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9c1ed132d54181998e7b5c957fc040'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.124180 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1345/2677] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:40 2022] DELETE /identity/v3/users/de9c1ed132d54181998e7b5c957fc040 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:41.392391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:41.393833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:41.409082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92807966-3992-4b4b-a41c-05fd1c00653d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:41.466679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:41.467318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.467597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.467870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] PATH_INFO: `/v3/projects/d8dcf6ded0504d2d8dc78c5028d4e13d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.469049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8dcf6ded0504d2d8dc78c5028d4e13d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:41.470871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:41.487529 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:41.489458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:41.503496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43dad4-ac97-4eec-a02d-61f0f269c855 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:41.508458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:41.509071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.509345 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.509671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] PATH_INFO: `/v3/users/626578b1a66740df871d037e1b6339ae` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.510796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=626578b1a66740df871d037e1b6339ae)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:41.512704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:41.518663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:41.522852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8dcf6ded0504d2d8dc78c5028d4e13d', 'request_id': 'req-3addbef1-ed13-4b96-b6ea-e628c83bccda'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.531702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8dcf6ded0504d2d8dc78c5028d4e13d', 'request_id': 'req-3addbef1-ed13-4b96-b6ea-e628c83bccda'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.540526 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8dcf6ded0504d2d8dc78c5028d4e13d', 'request_id': 'req-3addbef1-ed13-4b96-b6ea-e628c83bccda'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.551479 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] The token cache is being invalidate because project d8dcf6ded0504d2d8dc78c5028d4e13d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:41.551767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.552398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3addbef1-ed13-4b96-b6ea-e628c83bccda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.553891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1333/2678] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:41 2022] DELETE /identity/v3/projects/d8dcf6ded0504d2d8dc78c5028d4e13d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:41.559641 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2331c-0577-4e6b-b6da-eacb21b0c7d5 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.559919 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2331c-0577-4e6b-b6da-eacb21b0c7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.560279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2331c-0577-4e6b-b6da-eacb21b0c7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.563601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '626578b1a66740df871d037e1b6339ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.564331 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '626578b1a66740df871d037e1b6339ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.571723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '626578b1a66740df871d037e1b6339ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.572359 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '626578b1a66740df871d037e1b6339ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.579657 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-6fdae5e4-0021-4e9c-90c7-e7f853523285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '626578b1a66740df871d037e1b6339ae'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:41.580812 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a1b2331c-0577-4e6b-b6da-eacb21b0c7d5 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:41.582748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a1b2331c-0577-4e6b-b6da-eacb21b0c7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:41.600034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1346/2679] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:41 2022] DELETE /identity/v3/users/626578b1a66740df871d037e1b6339ae => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:41.604674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cf067-645f-4f2c-9ab1-2a4f8ac25a14 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.605039 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cf067-645f-4f2c-9ab1-2a4f8ac25a14 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.605390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cf067-645f-4f2c-9ab1-2a4f8ac25a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.624428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-343cf067-645f-4f2c-9ab1-2a4f8ac25a14 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:41.625669 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-343cf067-645f-4f2c-9ab1-2a4f8ac25a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:41.641743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1334/2680] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:41.649862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be553-3fc5-4427-b3c0-168d2238a3b7 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.650499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be553-3fc5-4427-b3c0-168d2238a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.650499 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be553-3fc5-4427-b3c0-168d2238a3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.671171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-d88be553-3fc5-4427-b3c0-168d2238a3b7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:41.673630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d88be553-3fc5-4427-b3c0-168d2238a3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:41.687393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1347/2681] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:41.697488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cf067-645f-4f2c-9ab1-2a4f8ac25a14 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:41.699119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-343cf067-645f-4f2c-9ab1-2a4f8ac25a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:41.718785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:41.743738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1335/2682] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:41.748036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88be553-3fc5-4427-b3c0-168d2238a3b7 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:41.757241 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88be553-3fc5-4427-b3c0-168d2238a3b7 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:41.761299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cf067-645f-4f2c-9ab1-2a4f8ac25a14 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:41.761943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392529fb-5b29-40a3-8ade-fbf8b198709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:41.762702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392529fb-5b29-40a3-8ade-fbf8b198709a admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.763176 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392529fb-5b29-40a3-8ade-fbf8b198709a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.763483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392529fb-5b29-40a3-8ade-fbf8b198709a admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.764110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392529fb-5b29-40a3-8ade-fbf8b198709a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:41.764427 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392529fb-5b29-40a3-8ade-fbf8b198709a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:41.765036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c4404-4e12-47ea-b039-0a3a3025121b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:41.765705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c4404-4e12-47ea-b039-0a3a3025121b service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.765985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c4404-4e12-47ea-b039-0a3a3025121b service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.766255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c4404-4e12-47ea-b039-0a3a3025121b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.766384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392529fb-5b29-40a3-8ade-fbf8b198709a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:41.775925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c4404-4e12-47ea-b039-0a3a3025121b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:41.777350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c4404-4e12-47ea-b039-0a3a3025121b service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:41.779182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1336/2683] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:29:41.783424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392529fb-5b29-40a3-8ade-fbf8b198709a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:41.787487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1348/2684] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:29:41.793219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392529fb-5b29-40a3-8ade-fbf8b198709a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:41.798185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd414-7539-4887-9900-65097b7e4dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:41.798771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd414-7539-4887-9900-65097b7e4dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.799045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd414-7539-4887-9900-65097b7e4dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.799314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd414-7539-4887-9900-65097b7e4dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.799930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dd414-7539-4887-9900-65097b7e4dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:41.801833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dd414-7539-4887-9900-65097b7e4dc8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:41.830094 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1337/2685] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:41 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:41.834838 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c4404-4e12-47ea-b039-0a3a3025121b service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:41.845227 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c4404-4e12-47ea-b039-0a3a3025121b service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:41.916107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fcd0f-6198-4102-8ba7-0a25cd109490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:41.916645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fcd0f-6198-4102-8ba7-0a25cd109490 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.916925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fcd0f-6198-4102-8ba7-0a25cd109490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.917201 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fcd0f-6198-4102-8ba7-0a25cd109490 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.917881 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fcd0f-6198-4102-8ba7-0a25cd109490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:41.919796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fcd0f-6198-4102-8ba7-0a25cd109490 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:41.944038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1349/2686] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:41 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:41.948074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd414-7539-4887-9900-65097b7e4dc8 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:41.959389 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd414-7539-4887-9900-65097b7e4dc8 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:41.963927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e65b2-cb5f-4e8b-a6b1-511455748943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:41.964522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e65b2-cb5f-4e8b-a6b1-511455748943 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:41.964790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e65b2-cb5f-4e8b-a6b1-511455748943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:41.965056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e65b2-cb5f-4e8b-a6b1-511455748943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:41.965712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e65b2-cb5f-4e8b-a6b1-511455748943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:41.967318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866e65b2-cb5f-4e8b-a6b1-511455748943 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:41.978010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1338/2687] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:41.982417 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fcd0f-6198-4102-8ba7-0a25cd109490 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:41.994287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fcd0f-6198-4102-8ba7-0a25cd109490 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:41.999224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04063f84-dc40-4e90-94ca-d543df285f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:41.999893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04063f84-dc40-4e90-94ca-d543df285f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.000173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04063f84-dc40-4e90-94ca-d543df285f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.000440 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04063f84-dc40-4e90-94ca-d543df285f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.001468 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04063f84-dc40-4e90-94ca-d543df285f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.002751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04063f84-dc40-4e90-94ca-d543df285f6f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.015001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1350/2688] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:42.018823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e65b2-cb5f-4e8b-a6b1-511455748943 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.029884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e65b2-cb5f-4e8b-a6b1-511455748943 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.034806 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381b86c-1cf6-48bf-9e3a-6a183dceb6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.035572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381b86c-1cf6-48bf-9e3a-6a183dceb6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.035908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381b86c-1cf6-48bf-9e3a-6a183dceb6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.036246 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381b86c-1cf6-48bf-9e3a-6a183dceb6b5 admin admin] PATH_INFO: `/v3/projects/62d41864e0714a0dbb3a512ab3f63a5b/users/dd8cc1113028491f82557d97a8d6a98a/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.061490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381b86c-1cf6-48bf-9e3a-6a183dceb6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d41864e0714a0dbb3a512ab3f63a5b, user_id=dd8cc1113028491f82557d97a8d6a98a, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.064842 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381b86c-1cf6-48bf-9e3a-6a183dceb6b5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.076088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04063f84-dc40-4e90-94ca-d543df285f6f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.077828 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-04063f84-dc40-4e90-94ca-d543df285f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:42.083956 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1339/2689] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:42 2022] PUT /identity/v3/projects/62d41864e0714a0dbb3a512ab3f63a5b/users/dd8cc1113028491f82557d97a8d6a98a/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:42.091265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04063f84-dc40-4e90-94ca-d543df285f6f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.094210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459a90b-59d4-4a96-8f63-21f27741a0ea None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.094434 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459a90b-59d4-4a96-8f63-21f27741a0ea None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.094955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459a90b-59d4-4a96-8f63-21f27741a0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.104703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:42.128116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-2459a90b-59d4-4a96-8f63-21f27741a0ea None None] MFA Rules not processed for user `dd8cc1113028491f82557d97a8d6a98a`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:42.129915 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2459a90b-59d4-4a96-8f63-21f27741a0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:42.152592 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.153242 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.153533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.153815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] PATH_INFO: `/v3/projects/386583f809004b959b61479ae52a4bce` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.154924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=386583f809004b959b61479ae52a4bce)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.156807 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.195489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1340/2690] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:42.202603 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:42.207081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459a90b-59d4-4a96-8f63-21f27741a0ea None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.208699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-2459a90b-59d4-4a96-8f63-21f27741a0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:42.210346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '386583f809004b959b61479ae52a4bce', 'request_id': 'req-3cce301d-aa65-4da9-b980-36e9a669f4ee'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:42.217793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '386583f809004b959b61479ae52a4bce', 'request_id': 'req-3cce301d-aa65-4da9-b980-36e9a669f4ee'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:42.224148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '386583f809004b959b61479ae52a4bce', 'request_id': 'req-3cce301d-aa65-4da9-b980-36e9a669f4ee'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:42.234414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] The token cache is being invalidate because project 386583f809004b959b61479ae52a4bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:42.234697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:42.235340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-3cce301d-aa65-4da9-b980-36e9a669f4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:42.236887 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1351/2691] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:42 2022] DELETE /identity/v3/projects/386583f809004b959b61479ae52a4bce => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:42.242260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54a2d1-941d-432e-ae53-f21fd7bde1df None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.242533 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54a2d1-941d-432e-ae53-f21fd7bde1df None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.242798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54a2d1-941d-432e-ae53-f21fd7bde1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.262164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-8e54a2d1-941d-432e-ae53-f21fd7bde1df None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:42.263451 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-8e54a2d1-941d-432e-ae53-f21fd7bde1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:42.269917 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459a90b-59d4-4a96-8f63-21f27741a0ea None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.317885 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f9fea-7feb-4919-b917-f4b9abefd1cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bafea2367e4c7992354bb2c23e7d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.318492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9fea-7feb-4919-b917-f4b9abefd1cb service glance] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.318767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9fea-7feb-4919-b917-f4b9abefd1cb service glance] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.319035 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f9fea-7feb-4919-b917-f4b9abefd1cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.320742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-811f9fea-7feb-4919-b917-f4b9abefd1cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:42.321505 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1352/2692] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:42.329050 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8f4f5-5225-4b3d-8924-e34cc80b45ff None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.329329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8f4f5-5225-4b3d-8924-e34cc80b45ff None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.329693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc8f4f5-5225-4b3d-8924-e34cc80b45ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.348671 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-cbc8f4f5-5225-4b3d-8924-e34cc80b45ff None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:42.349899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-cbc8f4f5-5225-4b3d-8924-e34cc80b45ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:42.357996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1353/2693] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:42.362012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8f4f5-5225-4b3d-8924-e34cc80b45ff None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.371854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc8f4f5-5225-4b3d-8924-e34cc80b45ff None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.376424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82985b-8973-48fa-bf7d-1337ade8f59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.377008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82985b-8973-48fa-bf7d-1337ade8f59b admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.377284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82985b-8973-48fa-bf7d-1337ade8f59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.377580 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82985b-8973-48fa-bf7d-1337ade8f59b admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.378219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82985b-8973-48fa-bf7d-1337ade8f59b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:29:42.378517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82985b-8973-48fa-bf7d-1337ade8f59b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.378693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f9fea-7feb-4919-b917-f4b9abefd1cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.380701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f9fea-7feb-4919-b917-f4b9abefd1cb service glance] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.380826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82985b-8973-48fa-bf7d-1337ade8f59b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.389482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1341/2694] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:29:42.392534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1354/2695] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:29:42.396843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f9fea-7feb-4919-b917-f4b9abefd1cb service glance] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.403359 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:42.407743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f9fea-7feb-4919-b917-f4b9abefd1cb service glance] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.414778 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c5478-d932-4921-b474-604fcadba4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.415386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c5478-d932-4921-b474-604fcadba4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.415654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c5478-d932-4921-b474-604fcadba4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.415923 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c5478-d932-4921-b474-604fcadba4e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.416555 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c5478-d932-4921-b474-604fcadba4e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.418620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c5478-d932-4921-b474-604fcadba4e1 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.447388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1342/2696] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:42 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:42.451428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82985b-8973-48fa-bf7d-1337ade8f59b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.461472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82985b-8973-48fa-bf7d-1337ade8f59b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.520704 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95046044-4d6f-479f-a265-022160ce31f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.521317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95046044-4d6f-479f-a265-022160ce31f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.521591 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95046044-4d6f-479f-a265-022160ce31f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.521859 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95046044-4d6f-479f-a265-022160ce31f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.522474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95046044-4d6f-479f-a265-022160ce31f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.524416 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95046044-4d6f-479f-a265-022160ce31f0 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.548491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1355/2697] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:42 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:42.549858 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c5478-d932-4921-b474-604fcadba4e1 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.561500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c5478-d932-4921-b474-604fcadba4e1 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.566181 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bfe33-92b2-420c-8bc1-f9780e39e527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.566898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bfe33-92b2-420c-8bc1-f9780e39e527 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.567542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bfe33-92b2-420c-8bc1-f9780e39e527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.567905 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bfe33-92b2-420c-8bc1-f9780e39e527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.568639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bfe33-92b2-420c-8bc1-f9780e39e527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.570354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bfe33-92b2-420c-8bc1-f9780e39e527 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.583194 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1343/2698] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:42.587250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95046044-4d6f-479f-a265-022160ce31f0 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.598437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95046044-4d6f-479f-a265-022160ce31f0 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.603583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c48ea4-4536-4743-920d-72c972d2eeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.604179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c48ea4-4536-4743-920d-72c972d2eeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.604454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c48ea4-4536-4743-920d-72c972d2eeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.604745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c48ea4-4536-4743-920d-72c972d2eeb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.605377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c48ea4-4536-4743-920d-72c972d2eeb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.607031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c48ea4-4536-4743-920d-72c972d2eeb2 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.619639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1356/2699] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:29:42.621978 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bfe33-92b2-420c-8bc1-f9780e39e527 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.632071 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bfe33-92b2-420c-8bc1-f9780e39e527 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.636837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2dab0-2b61-4400-bec3-779a4827b0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.637292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2dab0-2b61-4400-bec3-779a4827b0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.637601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2dab0-2b61-4400-bec3-779a4827b0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.637874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2dab0-2b61-4400-bec3-779a4827b0d8 admin admin] PATH_INFO: `/v3/projects/3277eaeb27724447a3864927021c2b79/users/b7b64272dd774de5a7cd22776df18eca/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.652082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2dab0-2b61-4400-bec3-779a4827b0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3277eaeb27724447a3864927021c2b79, user_id=b7b64272dd774de5a7cd22776df18eca, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.654799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2dab0-2b61-4400-bec3-779a4827b0d8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.670981 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1344/2700] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:42 2022] PUT /identity/v3/projects/3277eaeb27724447a3864927021c2b79/users/b7b64272dd774de5a7cd22776df18eca/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:42.676626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efc54a-8cb7-47b1-8302-c1148f4ec9fc None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.676961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efc54a-8cb7-47b1-8302-c1148f4ec9fc None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.677237 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efc54a-8cb7-47b1-8302-c1148f4ec9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.696424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-18efc54a-8cb7-47b1-8302-c1148f4ec9fc None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:42.697861 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-18efc54a-8cb7-47b1-8302-c1148f4ec9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:42.720798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:42.769997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1357/2701] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:42.780909 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2dab0-2b61-4400-bec3-779a4827b0d8 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:42.782868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7dd2dab0-2b61-4400-bec3-779a4827b0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:42.862841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2dab0-2b61-4400-bec3-779a4827b0d8 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:42.866615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47c836-9eff-4076-99d6-b86be4dde58b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:42.867293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47c836-9eff-4076-99d6-b86be4dde58b service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:42.867633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47c836-9eff-4076-99d6-b86be4dde58b service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:42.867970 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47c836-9eff-4076-99d6-b86be4dde58b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:42.878109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47c836-9eff-4076-99d6-b86be4dde58b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:42.879550 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47c836-9eff-4076-99d6-b86be4dde58b service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:42.890449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1345/2702] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:47.505552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b4e19-b0bf-442d-91b3-f3764d7902ee None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:47.506087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b4e19-b0bf-442d-91b3-f3764d7902ee None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:47.506087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b4e19-b0bf-442d-91b3-f3764d7902ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:47.570042 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-3e3b4e19-b0bf-442d-91b3-f3764d7902ee None None] MFA Rules not processed for user `b7b64272dd774de5a7cd22776df18eca`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:47.571285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3e3b4e19-b0bf-442d-91b3-f3764d7902ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:47.649236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1358/2703] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:47.653871 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47c836-9eff-4076-99d6-b86be4dde58b service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:47.665080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47c836-9eff-4076-99d6-b86be4dde58b service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:47.670757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b4343-a0aa-49a0-b05e-9a29d39cf51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:47.671503 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b4343-a0aa-49a0-b05e-9a29d39cf51b admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:47.671884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b4343-a0aa-49a0-b05e-9a29d39cf51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:47.672255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b4343-a0aa-49a0-b05e-9a29d39cf51b admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:47.673015 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b4343-a0aa-49a0-b05e-9a29d39cf51b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:47.675093 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b4343-a0aa-49a0-b05e-9a29d39cf51b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:47.712684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1346/2704] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:29:47 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:29:47.715159 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b4e19-b0bf-442d-91b3-f3764d7902ee None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:47.727379 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b4e19-b0bf-442d-91b3-f3764d7902ee None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:47.788030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee8241-3044-4a5b-b690-e8b1a19c1bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:47.788784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee8241-3044-4a5b-b690-e8b1a19c1bff admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:47.789099 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee8241-3044-4a5b-b690-e8b1a19c1bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:47.789384 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee8241-3044-4a5b-b690-e8b1a19c1bff admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:47.790279 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee8241-3044-4a5b-b690-e8b1a19c1bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:47.792633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee8241-3044-4a5b-b690-e8b1a19c1bff admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:47.815933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1359/2705] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:29:47 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:29:47.820862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b4343-a0aa-49a0-b05e-9a29d39cf51b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:47.832260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b4343-a0aa-49a0-b05e-9a29d39cf51b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:47.840572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f0527-e175-417c-b070-af3ccc8d301c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:47.841215 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f0527-e175-417c-b070-af3ccc8d301c admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:47.841494 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f0527-e175-417c-b070-af3ccc8d301c admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:47.841816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f0527-e175-417c-b070-af3ccc8d301c admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:47.842478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f0527-e175-417c-b070-af3ccc8d301c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:47.844096 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f0527-e175-417c-b070-af3ccc8d301c admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:47.855519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1347/2706] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:47.859848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee8241-3044-4a5b-b690-e8b1a19c1bff admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:47.883848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee8241-3044-4a5b-b690-e8b1a19c1bff admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:47.889095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf3013-a59e-4b66-98d8-541f410dc73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:47.889800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf3013-a59e-4b66-98d8-541f410dc73a admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:47.890081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf3013-a59e-4b66-98d8-541f410dc73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:47.890350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf3013-a59e-4b66-98d8-541f410dc73a admin admin] PATH_INFO: `/v3/projects/5ccbddcb715647b684259719d1429890/users/ad62e3b433094556b45ec3b2e13118a2/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:47.913265 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf3013-a59e-4b66-98d8-541f410dc73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccbddcb715647b684259719d1429890, user_id=ad62e3b433094556b45ec3b2e13118a2, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:47.916355 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf3013-a59e-4b66-98d8-541f410dc73a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:47.932117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1360/2707] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:47 2022] PUT /identity/v3/projects/5ccbddcb715647b684259719d1429890/users/ad62e3b433094556b45ec3b2e13118a2/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:47.936302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f0527-e175-417c-b070-af3ccc8d301c admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:47.953558 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f0527-e175-417c-b070-af3ccc8d301c admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:48.028052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c075e-fcd1-4803-b7d7-891ccf041045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:48.028052 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c075e-fcd1-4803-b7d7-891ccf041045 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:48.028399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c075e-fcd1-4803-b7d7-891ccf041045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:48.028481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c075e-fcd1-4803-b7d7-891ccf041045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:48.029087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c075e-fcd1-4803-b7d7-891ccf041045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:48.030707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c075e-fcd1-4803-b7d7-891ccf041045 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:48.041921 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1348/2708] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:29:48.046599 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf3013-a59e-4b66-98d8-541f410dc73a admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:48.058406 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf3013-a59e-4b66-98d8-541f410dc73a admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:48.063512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2ad1b-8f20-49fe-bb2a-95731ca5f0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:48.064238 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2ad1b-8f20-49fe-bb2a-95731ca5f0ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:48.064527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2ad1b-8f20-49fe-bb2a-95731ca5f0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:48.064804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2ad1b-8f20-49fe-bb2a-95731ca5f0ac admin admin] PATH_INFO: `/v3/projects/5ccbddcb715647b684259719d1429890/users/ad62e3b433094556b45ec3b2e13118a2/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:48.067801 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2ad1b-8f20-49fe-bb2a-95731ca5f0ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccbddcb715647b684259719d1429890, user_id=ad62e3b433094556b45ec3b2e13118a2, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:48.070545 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2ad1b-8f20-49fe-bb2a-95731ca5f0ac admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:48.083781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1361/2709] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:29:48 2022] PUT /identity/v3/projects/5ccbddcb715647b684259719d1429890/users/ad62e3b433094556b45ec3b2e13118a2/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:29:53.692436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7826a5-5280-4a7e-9204-03bae2e9c980 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:53.694201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7826a5-5280-4a7e-9204-03bae2e9c980 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:53.694707 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7826a5-5280-4a7e-9204-03bae2e9c980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:53.728584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:53.756544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-1b7826a5-5280-4a7e-9204-03bae2e9c980 None None] MFA Rules not processed for user `ad62e3b433094556b45ec3b2e13118a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:53.757938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1b7826a5-5280-4a7e-9204-03bae2e9c980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:53.890020 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1349/2710] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:53.905220 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2ad1b-8f20-49fe-bb2a-95731ca5f0ac admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:53.907093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-faf2ad1b-8f20-49fe-bb2a-95731ca5f0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:53.925763 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:54.059556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2ad1b-8f20-49fe-bb2a-95731ca5f0ac admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:54.065604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d837055-c43f-467a-9368-a6b1d4f8763f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:54.066914 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d837055-c43f-467a-9368-a6b1d4f8763f service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:54.067444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d837055-c43f-467a-9368-a6b1d4f8763f service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:54.067722 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d837055-c43f-467a-9368-a6b1d4f8763f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:54.131893 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d837055-c43f-467a-9368-a6b1d4f8763f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:54.133570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d837055-c43f-467a-9368-a6b1d4f8763f service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:54.143956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1362/2711] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:55.108103 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7826a5-5280-4a7e-9204-03bae2e9c980 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:55.221943 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7826a5-5280-4a7e-9204-03bae2e9c980 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:55.230622 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:55.231962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:55.231962 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:55.232347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:55.234291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:55.366370 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:55.368401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:55.377272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:55.398793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f69046-8250-4788-909e-3aef4dcc59d5 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:55.399464 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f69046-8250-4788-909e-3aef4dcc59d5 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:55.399567 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1350/2712] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:55.399666 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f69046-8250-4788-909e-3aef4dcc59d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:55.423369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-37f69046-8250-4788-909e-3aef4dcc59d5 None None] MFA Rules not processed for user `b7b64272dd774de5a7cd22776df18eca`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:55.424742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-37f69046-8250-4788-909e-3aef4dcc59d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:55.433872 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1363/2713] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:29:58.454470 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:58.456082 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:58.473171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c86ad7-0935-4174-89d4-2762ad26e9f8 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:58.479797 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:58.480793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:58.481408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:58.481914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] PATH_INFO: `/v3/users/dd8cc1113028491f82557d97a8d6a98a` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:58.483284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd8cc1113028491f82557d97a8d6a98a)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:58.485478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:58.579796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8cc1113028491f82557d97a8d6a98a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:58.580507 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8cc1113028491f82557d97a8d6a98a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:58.591765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8cc1113028491f82557d97a8d6a98a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:58.608222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8cc1113028491f82557d97a8d6a98a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:58.609022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8cc1113028491f82557d97a8d6a98a'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:58.661616 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1351/2714] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:58 2022] DELETE /identity/v3/users/dd8cc1113028491f82557d97a8d6a98a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:58.665667 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676a103-2e71-4d55-b2f2-da588d883f06 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:58.666496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676a103-2e71-4d55-b2f2-da588d883f06 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:58.666915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676a103-2e71-4d55-b2f2-da588d883f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:58.721236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1676a103-2e71-4d55-b2f2-da588d883f06 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:58.724329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1676a103-2e71-4d55-b2f2-da588d883f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:58.753638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=100 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:58.812273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1364/2715] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:29:58.824880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:58.826906 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:59.032286 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c103324-d0c8-4d7a-b430-c0d9b0b0b222 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:59.043009 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b3391-4341-4495-888e-945662116c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:59.052935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3391-4341-4495-888e-945662116c25 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:59.054914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3391-4341-4495-888e-945662116c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:59.055195 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b3391-4341-4495-888e-945662116c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:59.081994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b3391-4341-4495-888e-945662116c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:59.089022 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b3391-4341-4495-888e-945662116c25 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:59.120980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1352/2716] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:29:59.235449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676a103-2e71-4d55-b2f2-da588d883f06 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:59.237168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1676a103-2e71-4d55-b2f2-da588d883f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:59.250618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676a103-2e71-4d55-b2f2-da588d883f06 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:59.256043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:59.256730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:59.257055 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:59.257333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] PATH_INFO: `/v3/users/10d2803e109849fda2828f66f51b8a0c` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:59.258520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d2803e109849fda2828f66f51b8a0c)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:59.260480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:59.314552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d2803e109849fda2828f66f51b8a0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.315463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d2803e109849fda2828f66f51b8a0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.325185 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d2803e109849fda2828f66f51b8a0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.326035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d2803e109849fda2828f66f51b8a0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.337534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d2803e109849fda2828f66f51b8a0c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.361622 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1365/2717] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:29:59 2022] DELETE /identity/v3/users/10d2803e109849fda2828f66f51b8a0c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:59.576860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b3391-4341-4495-888e-945662116c25 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:59.579518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e47b3391-4341-4495-888e-945662116c25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:59.630373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b3391-4341-4495-888e-945662116c25 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:59.661588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:29:59.666690 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:59.740308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:59.741355 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:59.741730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:59.742070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] PATH_INFO: `/v3/projects/62d41864e0714a0dbb3a512ab3f63a5b` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:59.743343 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d41864e0714a0dbb3a512ab3f63a5b)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:59.745445 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:59.748251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:59.754413 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41e0dc-6f4d-41d4-9459-5c8249651b24 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:29:59.760166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:29:59.760822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:59.761150 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:59.761427 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] PATH_INFO: `/v3/projects/05398ef99be34b57b61052a0f07f57f8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:59.762644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=05398ef99be34b57b61052a0f07f57f8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:29:59.764536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:29:59.808661 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d41864e0714a0dbb3a512ab3f63a5b', 'request_id': 'req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.818654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d41864e0714a0dbb3a512ab3f63a5b', 'request_id': 'req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.822640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:29:59.826087 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d41864e0714a0dbb3a512ab3f63a5b', 'request_id': 'req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.829924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05398ef99be34b57b61052a0f07f57f8', 'request_id': 'req-fba3a3e9-ef88-4805-8352-dcb07fa327fc'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.839212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] The token cache is being invalidate because project 62d41864e0714a0dbb3a512ab3f63a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:59.839583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.840677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-803f5785-78fd-41ff-b0fe-ec09f9e75bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.842700 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05398ef99be34b57b61052a0f07f57f8', 'request_id': 'req-fba3a3e9-ef88-4805-8352-dcb07fa327fc'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.843801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1353/2718] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:59 2022] DELETE /identity/v3/projects/62d41864e0714a0dbb3a512ab3f63a5b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:59.849208 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522e04f-fddf-47a6-905d-fa3ecd1b7c3c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:59.849535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522e04f-fddf-47a6-905d-fa3ecd1b7c3c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:59.849795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522e04f-fddf-47a6-905d-fa3ecd1b7c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:59.858873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05398ef99be34b57b61052a0f07f57f8', 'request_id': 'req-fba3a3e9-ef88-4805-8352-dcb07fa327fc'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.871641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] The token cache is being invalidate because project 05398ef99be34b57b61052a0f07f57f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:29:59.872259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.872996 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-fba3a3e9-ef88-4805-8352-dcb07fa327fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:29:59.874853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1366/2719] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:29:59 2022] DELETE /identity/v3/projects/05398ef99be34b57b61052a0f07f57f8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:29:59.880221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49740853-fac7-474a-92cd-8ae7dae92ee1 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:29:59.880743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49740853-fac7-474a-92cd-8ae7dae92ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:29:59.881267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49740853-fac7-474a-92cd-8ae7dae92ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:29:59.915849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-3522e04f-fddf-47a6-905d-fa3ecd1b7c3c None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:59.917681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3522e04f-fddf-47a6-905d-fa3ecd1b7c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:29:59.925724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-49740853-fac7-474a-92cd-8ae7dae92ee1 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:29:59.927171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-49740853-fac7-474a-92cd-8ae7dae92ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:00.005071 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1354/2720] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:00.013013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b9963-13b7-4d2b-9798-ee9808ec653e None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.013294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b9963-13b7-4d2b-9798-ee9808ec653e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.013618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b9963-13b7-4d2b-9798-ee9808ec653e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.023117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1367/2721] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:00.031682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842548d4-6769-4198-9a56-894be012d7d3 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.032797 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842548d4-6769-4198-9a56-894be012d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.033139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842548d4-6769-4198-9a56-894be012d7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.038803 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-e89b9963-13b7-4d2b-9798-ee9808ec653e None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:00.040544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e89b9963-13b7-4d2b-9798-ee9808ec653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:00.050028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1355/2722] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:00.054330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b9963-13b7-4d2b-9798-ee9808ec653e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.061329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-842548d4-6769-4198-9a56-894be012d7d3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:00.062595 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-842548d4-6769-4198-9a56-894be012d7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:00.064819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b9963-13b7-4d2b-9798-ee9808ec653e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.070748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff9c3-f998-4f72-8254-4dffc89851e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.071428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff9c3-f998-4f72-8254-4dffc89851e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.071708 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff9c3-f998-4f72-8254-4dffc89851e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.073655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1368/2723] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:00.073757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ff9c3-f998-4f72-8254-4dffc89851e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.073757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ff9c3-f998-4f72-8254-4dffc89851e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:30:00.073757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ff9c3-f998-4f72-8254-4dffc89851e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.074691 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ff9c3-f998-4f72-8254-4dffc89851e9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.075755 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842548d4-6769-4198-9a56-894be012d7d3 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.088491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1356/2724] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:30:00.093572 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842548d4-6769-4198-9a56-894be012d7d3 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.093784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff9c3-f998-4f72-8254-4dffc89851e9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.103485 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875746e4-acd0-4997-aa27-51595514632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.104186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ff9c3-f998-4f72-8254-4dffc89851e9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.104325 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875746e4-acd0-4997-aa27-51595514632e admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.104476 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875746e4-acd0-4997-aa27-51595514632e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.104723 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875746e4-acd0-4997-aa27-51595514632e admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.105491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875746e4-acd0-4997-aa27-51595514632e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:30:00.105912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875746e4-acd0-4997-aa27-51595514632e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.107510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875746e4-acd0-4997-aa27-51595514632e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.108890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef696952-a99d-4b6d-a830-404b450778a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.109595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef696952-a99d-4b6d-a830-404b450778a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.109978 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef696952-a99d-4b6d-a830-404b450778a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.110274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef696952-a99d-4b6d-a830-404b450778a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.110974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef696952-a99d-4b6d-a830-404b450778a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.113637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef696952-a99d-4b6d-a830-404b450778a9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.126182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1369/2725] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 20:30:00.127727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875746e4-acd0-4997-aa27-51595514632e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.138369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875746e4-acd0-4997-aa27-51595514632e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.142959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2499165-e8d5-4954-b604-d19a6b991a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.143555 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2499165-e8d5-4954-b604-d19a6b991a84 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.143862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2499165-e8d5-4954-b604-d19a6b991a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.144131 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2499165-e8d5-4954-b604-d19a6b991a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.144759 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2499165-e8d5-4954-b604-d19a6b991a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.144887 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1357/2726] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:00.146687 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2499165-e8d5-4954-b604-d19a6b991a84 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.148644 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef696952-a99d-4b6d-a830-404b450778a9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.173915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1370/2727] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:00.178204 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2499165-e8d5-4954-b604-d19a6b991a84 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.181634 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:00.232501 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef696952-a99d-4b6d-a830-404b450778a9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.247164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2499165-e8d5-4954-b604-d19a6b991a84 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.253741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a27cc-d227-401a-a179-f4b46b384050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.254837 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a27cc-d227-401a-a179-f4b46b384050 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.255119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a27cc-d227-401a-a179-f4b46b384050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.255390 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a27cc-d227-401a-a179-f4b46b384050 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.256510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a27cc-d227-401a-a179-f4b46b384050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.258917 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a27cc-d227-401a-a179-f4b46b384050 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.288351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1371/2728] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:00.292617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a27cc-d227-401a-a179-f4b46b384050 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.297554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:00.300841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01a2c9-c7fb-4b89-964a-5065363d3507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.301653 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01a2c9-c7fb-4b89-964a-5065363d3507 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.301945 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01a2c9-c7fb-4b89-964a-5065363d3507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.302236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01a2c9-c7fb-4b89-964a-5065363d3507 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.302855 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01a2c9-c7fb-4b89-964a-5065363d3507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.303488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a27cc-d227-401a-a179-f4b46b384050 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.304837 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01a2c9-c7fb-4b89-964a-5065363d3507 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.309148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694221c-e4c4-4028-853e-ae28a0a2aef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.310215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694221c-e4c4-4028-853e-ae28a0a2aef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.310658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694221c-e4c4-4028-853e-ae28a0a2aef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.311153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694221c-e4c4-4028-853e-ae28a0a2aef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.312030 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694221c-e4c4-4028-853e-ae28a0a2aef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.313684 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694221c-e4c4-4028-853e-ae28a0a2aef6 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.330416 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1358/2729] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:00.332516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1372/2730] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:00.334738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01a2c9-c7fb-4b89-964a-5065363d3507 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.336181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694221c-e4c4-4028-853e-ae28a0a2aef6 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.344863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01a2c9-c7fb-4b89-964a-5065363d3507 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.346682 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694221c-e4c4-4028-853e-ae28a0a2aef6 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.349408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d94c54-6be4-4039-a20f-c8fa09af622e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.350059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d94c54-6be4-4039-a20f-c8fa09af622e admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.350330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d94c54-6be4-4039-a20f-c8fa09af622e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.350619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d94c54-6be4-4039-a20f-c8fa09af622e admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.351232 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d94c54-6be4-4039-a20f-c8fa09af622e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.351428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a14586-12a5-448e-90f2-88569040caed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.352006 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a14586-12a5-448e-90f2-88569040caed admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.352286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a14586-12a5-448e-90f2-88569040caed admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.352559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a14586-12a5-448e-90f2-88569040caed admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.352788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d94c54-6be4-4039-a20f-c8fa09af622e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.353191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a14586-12a5-448e-90f2-88569040caed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.354824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a14586-12a5-448e-90f2-88569040caed admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.366023 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1359/2731] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:30:00.366191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1373/2732] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:30:00.367329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d94c54-6be4-4039-a20f-c8fa09af622e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.369934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a14586-12a5-448e-90f2-88569040caed admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.377383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d94c54-6be4-4039-a20f-c8fa09af622e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.380025 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a14586-12a5-448e-90f2-88569040caed admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.382531 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f7685-3a07-49e2-b14e-b4cc9b82df02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.383139 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f7685-3a07-49e2-b14e-b4cc9b82df02 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.383373 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f7685-3a07-49e2-b14e-b4cc9b82df02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.383645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f7685-3a07-49e2-b14e-b4cc9b82df02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.384261 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f7685-3a07-49e2-b14e-b4cc9b82df02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.384637 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491fe5e-2ff0-4b58-9fe4-a99588ed53aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.385262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491fe5e-2ff0-4b58-9fe4-a99588ed53aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.385564 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491fe5e-2ff0-4b58-9fe4-a99588ed53aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.385842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491fe5e-2ff0-4b58-9fe4-a99588ed53aa admin admin] PATH_INFO: `/v3/projects/440bef4d416a40509f6da17982d0dbe4/users/83bd431253c745aea2e25a2ba5504a26/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.385952 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f7685-3a07-49e2-b14e-b4cc9b82df02 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.396212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1360/2733] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:30:00.400185 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f7685-3a07-49e2-b14e-b4cc9b82df02 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.403831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491fe5e-2ff0-4b58-9fe4-a99588ed53aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=440bef4d416a40509f6da17982d0dbe4, user_id=83bd431253c745aea2e25a2ba5504a26, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.407232 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491fe5e-2ff0-4b58-9fe4-a99588ed53aa admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.410329 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f7685-3a07-49e2-b14e-b4cc9b82df02 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.415294 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa270cbc-ed54-4a5d-84f0-3818996344b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.415973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa270cbc-ed54-4a5d-84f0-3818996344b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.416250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa270cbc-ed54-4a5d-84f0-3818996344b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.416515 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa270cbc-ed54-4a5d-84f0-3818996344b5 admin admin] PATH_INFO: `/v3/projects/cb23c05314cb497688b15e86f4a99d9a/users/bef3afe7e66447ac9f603a68d5b5cac7/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.422215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1374/2734] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:00 2022] PUT /identity/v3/projects/440bef4d416a40509f6da17982d0dbe4/users/83bd431253c745aea2e25a2ba5504a26/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:00.434435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6c7f3-0a70-4db5-93bb-8041f5bd0f0a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.434658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6c7f3-0a70-4db5-93bb-8041f5bd0f0a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.434960 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6c7f3-0a70-4db5-93bb-8041f5bd0f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.438544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa270cbc-ed54-4a5d-84f0-3818996344b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb23c05314cb497688b15e86f4a99d9a, user_id=bef3afe7e66447ac9f603a68d5b5cac7, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.442034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa270cbc-ed54-4a5d-84f0-3818996344b5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.457262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1361/2735] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:00 2022] PUT /identity/v3/projects/cb23c05314cb497688b15e86f4a99d9a/users/bef3afe7e66447ac9f603a68d5b5cac7/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:30:00.466342 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-9af6c7f3-0a70-4db5-93bb-8041f5bd0f0a None None] MFA Rules not processed for user `83bd431253c745aea2e25a2ba5504a26`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:00.467284 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3a5e-a828-496c-aa5a-e81e366d1ede None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.467565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9af6c7f3-0a70-4db5-93bb-8041f5bd0f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:00.467671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3a5e-a828-496c-aa5a-e81e366d1ede None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.467833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab3a5e-a828-496c-aa5a-e81e366d1ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.495702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4eab3a5e-a828-496c-aa5a-e81e366d1ede None None] MFA Rules not processed for user `bef3afe7e66447ac9f603a68d5b5cac7`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:00.496895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4eab3a5e-a828-496c-aa5a-e81e366d1ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:00.532478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1375/2736] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:00.537715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6c7f3-0a70-4db5-93bb-8041f5bd0f0a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.574485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1362/2737] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:00.586351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3a5e-a828-496c-aa5a-e81e366d1ede None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.587976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4eab3a5e-a828-496c-aa5a-e81e366d1ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:00.598304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:00.607356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6c7f3-0a70-4db5-93bb-8041f5bd0f0a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.611972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d11a1-9e0c-410d-b968-38fd5f2188a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.613166 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d11a1-9e0c-410d-b968-38fd5f2188a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.613441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d11a1-9e0c-410d-b968-38fd5f2188a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.613738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d11a1-9e0c-410d-b968-38fd5f2188a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.614368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d11a1-9e0c-410d-b968-38fd5f2188a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.616261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d11a1-9e0c-410d-b968-38fd5f2188a6 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.647604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1376/2738] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:00.648888 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d11a1-9e0c-410d-b968-38fd5f2188a6 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.649170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab3a5e-a828-496c-aa5a-e81e366d1ede None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.687699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:00.705408 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4145786-5344-4a85-9506-fcb1e60a5e7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bafea2367e4c7992354bb2c23e7d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.706046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4145786-5344-4a85-9506-fcb1e60a5e7c service glance] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.706345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4145786-5344-4a85-9506-fcb1e60a5e7c service glance] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.706621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4145786-5344-4a85-9506-fcb1e60a5e7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.719067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d11a1-9e0c-410d-b968-38fd5f2188a6 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.724149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9e37a-324b-4e87-97ab-964fde711b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.724747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9e37a-324b-4e87-97ab-964fde711b06 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.725018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9e37a-324b-4e87-97ab-964fde711b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.725282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9e37a-324b-4e87-97ab-964fde711b06 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.725934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9e37a-324b-4e87-97ab-964fde711b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.727833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9e37a-324b-4e87-97ab-964fde711b06 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.751129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1377/2739] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:00 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:00.755188 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9e37a-324b-4e87-97ab-964fde711b06 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.764497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9e37a-324b-4e87-97ab-964fde711b06 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.765573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4145786-5344-4a85-9506-fcb1e60a5e7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.767005 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4145786-5344-4a85-9506-fcb1e60a5e7c service glance] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.769113 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f2066-6f97-426a-9d8d-d077903cfb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.769719 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f2066-6f97-426a-9d8d-d077903cfb92 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.769997 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f2066-6f97-426a-9d8d-d077903cfb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.770270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f2066-6f97-426a-9d8d-d077903cfb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.770905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f2066-6f97-426a-9d8d-d077903cfb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.772526 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f2066-6f97-426a-9d8d-d077903cfb92 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.779345 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1363/2740] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:30:00.783794 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1378/2741] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:30:00.789487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4145786-5344-4a85-9506-fcb1e60a5e7c service glance] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.800517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4145786-5344-4a85-9506-fcb1e60a5e7c service glance] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.805478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a5a3a-6a4c-4e14-9aed-c7d6ddda4262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.805937 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a5a3a-6a4c-4e14-9aed-c7d6ddda4262 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.806212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a5a3a-6a4c-4e14-9aed-c7d6ddda4262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.806481 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a5a3a-6a4c-4e14-9aed-c7d6ddda4262 admin admin] PATH_INFO: `/v3/projects/f338138030e64410b7e8ae071213bd81/users/8ec3deebd60f4d78b24dbfd132172bb2/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.822435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a5a3a-6a4c-4e14-9aed-c7d6ddda4262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f338138030e64410b7e8ae071213bd81, user_id=8ec3deebd60f4d78b24dbfd132172bb2, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.825081 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a5a3a-6a4c-4e14-9aed-c7d6ddda4262 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.839930 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1364/2742] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:00 2022] PUT /identity/v3/projects/f338138030e64410b7e8ae071213bd81/users/8ec3deebd60f4d78b24dbfd132172bb2/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:00.844267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f2066-6f97-426a-9d8d-d077903cfb92 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.866568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f2066-6f97-426a-9d8d-d077903cfb92 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.957730 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f606be5-d665-4baa-ab6d-de8ec2716e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.958496 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606be5-d665-4baa-ab6d-de8ec2716e24 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.958904 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606be5-d665-4baa-ab6d-de8ec2716e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.959183 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606be5-d665-4baa-ab6d-de8ec2716e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.959829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f606be5-d665-4baa-ab6d-de8ec2716e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.961424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f606be5-d665-4baa-ab6d-de8ec2716e24 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:00.973562 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1379/2743] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:00.976723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a5a3a-6a4c-4e14-9aed-c7d6ddda4262 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:00.987712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a5a3a-6a4c-4e14-9aed-c7d6ddda4262 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:00.992759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d820ef4-1e28-41b8-b785-830f895c0c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:00.993639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820ef4-1e28-41b8-b785-830f895c0c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:00.993983 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820ef4-1e28-41b8-b785-830f895c0c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:00.994260 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820ef4-1e28-41b8-b785-830f895c0c83 admin admin] PATH_INFO: `/v3/projects/f338138030e64410b7e8ae071213bd81/users/8ec3deebd60f4d78b24dbfd132172bb2/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:00.996292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d820ef4-1e28-41b8-b785-830f895c0c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f338138030e64410b7e8ae071213bd81, user_id=8ec3deebd60f4d78b24dbfd132172bb2, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:00.999034 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d820ef4-1e28-41b8-b785-830f895c0c83 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:01.010583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1365/2744] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:00 2022] PUT /identity/v3/projects/f338138030e64410b7e8ae071213bd81/users/8ec3deebd60f4d78b24dbfd132172bb2/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:01.024514 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c21c-49a3-4527-9a3f-259a99d114f9 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:01.024829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c21c-49a3-4527-9a3f-259a99d114f9 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:01.025107 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674c21c-49a3-4527-9a3f-259a99d114f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:01.059034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e674c21c-49a3-4527-9a3f-259a99d114f9 None None] MFA Rules not processed for user `8ec3deebd60f4d78b24dbfd132172bb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:01.060638 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e674c21c-49a3-4527-9a3f-259a99d114f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:01.135674 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1380/2745] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:01.159351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d820ef4-1e28-41b8-b785-830f895c0c83 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:01.161911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4d820ef4-1e28-41b8-b785-830f895c0c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:01.208703 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:01.224301 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d820ef4-1e28-41b8-b785-830f895c0c83 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:01.228318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28648757-94a3-4772-bb9a-9c15f35f716e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:01.229268 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28648757-94a3-4772-bb9a-9c15f35f716e service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:01.229636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28648757-94a3-4772-bb9a-9c15f35f716e service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:01.229985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28648757-94a3-4772-bb9a-9c15f35f716e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:01.239610 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28648757-94a3-4772-bb9a-9c15f35f716e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:01.241184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28648757-94a3-4772-bb9a-9c15f35f716e service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:01.251679 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1366/2746] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:01.683783 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c21c-49a3-4527-9a3f-259a99d114f9 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:01.726398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:01.760766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674c21c-49a3-4527-9a3f-259a99d114f9 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:01.765339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:01.765995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:01.766273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:01.766541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] PATH_INFO: `/v3/users/83bd431253c745aea2e25a2ba5504a26` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:01.767643 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bd431253c745aea2e25a2ba5504a26)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:01.769579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:01.818017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bd431253c745aea2e25a2ba5504a26'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:01.818663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bd431253c745aea2e25a2ba5504a26'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:01.825868 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bd431253c745aea2e25a2ba5504a26'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:01.826499 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bd431253c745aea2e25a2ba5504a26'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:01.833847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bd431253c745aea2e25a2ba5504a26'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:01.850450 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1381/2747] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:01 2022] DELETE /identity/v3/users/83bd431253c745aea2e25a2ba5504a26 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:01.854898 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6893286-ea48-4d2f-af1c-ac3379ccf72a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:01.855356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6893286-ea48-4d2f-af1c-ac3379ccf72a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:01.855472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6893286-ea48-4d2f-af1c-ac3379ccf72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:01.877430 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-a6893286-ea48-4d2f-af1c-ac3379ccf72a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:01.879399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a6893286-ea48-4d2f-af1c-ac3379ccf72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:01.943016 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1367/2748] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:01.952709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:01.954158 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:02.023570 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858e216-8b94-46b4-8390-f51999c37bd0 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:02.027488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff783b-ec53-4d90-8e26-0d1c50c6c61a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:02.028094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff783b-ec53-4d90-8e26-0d1c50c6c61a service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:02.028375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff783b-ec53-4d90-8e26-0d1c50c6c61a service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:02.028647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff783b-ec53-4d90-8e26-0d1c50c6c61a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:02.039805 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff783b-ec53-4d90-8e26-0d1c50c6c61a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:02.041244 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff783b-ec53-4d90-8e26-0d1c50c6c61a service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:02.051430 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1382/2749] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:02.344449 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6893286-ea48-4d2f-af1c-ac3379ccf72a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:02.349250 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-a6893286-ea48-4d2f-af1c-ac3379ccf72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:02.380704 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6893286-ea48-4d2f-af1c-ac3379ccf72a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:02.401435 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:02.402761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:02.402761 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:02.404744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] PATH_INFO: `/v3/projects/440bef4d416a40509f6da17982d0dbe4` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:02.406497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=440bef4d416a40509f6da17982d0dbe4)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:02.411649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:02.476210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440bef4d416a40509f6da17982d0dbe4', 'request_id': 'req-5c97da8c-569d-43dc-bb6b-ebaee96721f0'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.483929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440bef4d416a40509f6da17982d0dbe4', 'request_id': 'req-5c97da8c-569d-43dc-bb6b-ebaee96721f0'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.490692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440bef4d416a40509f6da17982d0dbe4', 'request_id': 'req-5c97da8c-569d-43dc-bb6b-ebaee96721f0'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.502212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] The token cache is being invalidate because project 440bef4d416a40509f6da17982d0dbe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:02.502518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.503180 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.504684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1368/2750] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:02 2022] DELETE /identity/v3/projects/440bef4d416a40509f6da17982d0dbe4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:02.508721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff783b-ec53-4d90-8e26-0d1c50c6c61a service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:02.518347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff783b-ec53-4d90-8e26-0d1c50c6c61a service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:02.567928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:02.578543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:02.579180 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:02.579454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:02.579773 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] PATH_INFO: `/v3/users/8ec3deebd60f4d78b24dbfd132172bb2` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:02.581019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec3deebd60f4d78b24dbfd132172bb2)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:02.583483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:02.639633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec3deebd60f4d78b24dbfd132172bb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.640341 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec3deebd60f4d78b24dbfd132172bb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.648646 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec3deebd60f4d78b24dbfd132172bb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.649368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec3deebd60f4d78b24dbfd132172bb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.656800 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-784b8e40-19ff-4bf4-adb0-210a0f2e7569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec3deebd60f4d78b24dbfd132172bb2'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:02.675977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1383/2751] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:02 2022] DELETE /identity/v3/users/8ec3deebd60f4d78b24dbfd132172bb2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:02.951267 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:02.952722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:02.967590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97da8c-569d-43dc-bb6b-ebaee96721f0 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.025614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.026259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.026535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.026807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] PATH_INFO: `/v3/projects/f338138030e64410b7e8ae071213bd81` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.028027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f338138030e64410b7e8ae071213bd81)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.029975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.038399 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:03.077903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f338138030e64410b7e8ae071213bd81', 'request_id': 'req-74c0e384-e487-4781-bd4e-da3073c99b74'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:03.085603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f338138030e64410b7e8ae071213bd81', 'request_id': 'req-74c0e384-e487-4781-bd4e-da3073c99b74'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:03.091773 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f338138030e64410b7e8ae071213bd81', 'request_id': 'req-74c0e384-e487-4781-bd4e-da3073c99b74'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:03.102072 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] The token cache is being invalidate because project f338138030e64410b7e8ae071213bd81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:03.102379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:03.102994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-74c0e384-e487-4781-bd4e-da3073c99b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:03.104441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1369/2752] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:02 2022] DELETE /identity/v3/projects/f338138030e64410b7e8ae071213bd81 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:03.109786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0afe9f-575e-4921-ae88-0995e0296302 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.110246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0afe9f-575e-4921-ae88-0995e0296302 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.110376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0afe9f-575e-4921-ae88-0995e0296302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.130551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-9b0afe9f-575e-4921-ae88-0995e0296302 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:03.131731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9b0afe9f-575e-4921-ae88-0995e0296302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:03.192019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1384/2753] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:03.201111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a267cd-0391-46ce-bdec-4e60ac31f0ee None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.201384 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a267cd-0391-46ce-bdec-4e60ac31f0ee None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.201734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a267cd-0391-46ce-bdec-4e60ac31f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.225149 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-92a267cd-0391-46ce-bdec-4e60ac31f0ee None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:03.226597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-92a267cd-0391-46ce-bdec-4e60ac31f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:03.235339 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1370/2754] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:03.240128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0afe9f-575e-4921-ae88-0995e0296302 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.249856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0afe9f-575e-4921-ae88-0995e0296302 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.255101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34302f9-bf6d-4173-8f11-fc0335b0ece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.255697 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34302f9-bf6d-4173-8f11-fc0335b0ece5 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.255975 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34302f9-bf6d-4173-8f11-fc0335b0ece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.256245 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34302f9-bf6d-4173-8f11-fc0335b0ece5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.256884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34302f9-bf6d-4173-8f11-fc0335b0ece5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:30:03.257197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34302f9-bf6d-4173-8f11-fc0335b0ece5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.258816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34302f9-bf6d-4173-8f11-fc0335b0ece5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.270045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1385/2755] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:30:03.274113 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a267cd-0391-46ce-bdec-4e60ac31f0ee None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.283074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a267cd-0391-46ce-bdec-4e60ac31f0ee None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.287835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1c0fe-b5d8-4fa6-9725-a0bc7becbd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.288437 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1c0fe-b5d8-4fa6-9725-a0bc7becbd14 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.288711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1c0fe-b5d8-4fa6-9725-a0bc7becbd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.288976 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1c0fe-b5d8-4fa6-9725-a0bc7becbd14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.289638 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1c0fe-b5d8-4fa6-9725-a0bc7becbd14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.291561 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1c0fe-b5d8-4fa6-9725-a0bc7becbd14 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.319290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1371/2756] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:03 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:03.323596 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34302f9-bf6d-4173-8f11-fc0335b0ece5 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.333632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34302f9-bf6d-4173-8f11-fc0335b0ece5 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.388691 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd3426-f49a-47a5-8cd9-6db906337922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.389287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd3426-f49a-47a5-8cd9-6db906337922 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.389588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd3426-f49a-47a5-8cd9-6db906337922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.389860 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd3426-f49a-47a5-8cd9-6db906337922 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.390493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd3426-f49a-47a5-8cd9-6db906337922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.392534 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd3426-f49a-47a5-8cd9-6db906337922 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.400535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:03.416287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1386/2757] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:03 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:03.420361 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1c0fe-b5d8-4fa6-9725-a0bc7becbd14 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.434270 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1c0fe-b5d8-4fa6-9725-a0bc7becbd14 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.439386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bc8d-1066-4860-8488-0eeaef8cea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.440109 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bc8d-1066-4860-8488-0eeaef8cea99 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.442672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bc8d-1066-4860-8488-0eeaef8cea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.442672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bc8d-1066-4860-8488-0eeaef8cea99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.442672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6bc8d-1066-4860-8488-0eeaef8cea99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.444107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6bc8d-1066-4860-8488-0eeaef8cea99 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.460119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1372/2758] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:03.461051 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd3426-f49a-47a5-8cd9-6db906337922 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.473374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd3426-f49a-47a5-8cd9-6db906337922 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.478796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30858459-e389-4bc1-8a4a-8f86a01167ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.479508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30858459-e389-4bc1-8a4a-8f86a01167ed admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.479905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30858459-e389-4bc1-8a4a-8f86a01167ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.480402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30858459-e389-4bc1-8a4a-8f86a01167ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.481385 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30858459-e389-4bc1-8a4a-8f86a01167ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.483187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30858459-e389-4bc1-8a4a-8f86a01167ed admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.502584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1387/2759] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:03.504051 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bc8d-1066-4860-8488-0eeaef8cea99 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.515404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bc8d-1066-4860-8488-0eeaef8cea99 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.519998 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d16fd1-67d5-43ae-b164-b027152f1bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.520650 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d16fd1-67d5-43ae-b164-b027152f1bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.520954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d16fd1-67d5-43ae-b164-b027152f1bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.521221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d16fd1-67d5-43ae-b164-b027152f1bb1 admin admin] PATH_INFO: `/v3/projects/9ae14f049fd645b6b7ce36192ab165d6/users/784549310ee74732838da7d18d82cc53/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.536097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d16fd1-67d5-43ae-b164-b027152f1bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ae14f049fd645b6b7ce36192ab165d6, user_id=784549310ee74732838da7d18d82cc53, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.538712 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d16fd1-67d5-43ae-b164-b027152f1bb1 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.551633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1373/2760] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:03 2022] PUT /identity/v3/projects/9ae14f049fd645b6b7ce36192ab165d6/users/784549310ee74732838da7d18d82cc53/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:30:03.561737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43133e88-47d5-4b13-af8c-596b8dc03d2b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.562045 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43133e88-47d5-4b13-af8c-596b8dc03d2b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.562317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43133e88-47d5-4b13-af8c-596b8dc03d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.589402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-43133e88-47d5-4b13-af8c-596b8dc03d2b None None] MFA Rules not processed for user `784549310ee74732838da7d18d82cc53`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:03.590688 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-43133e88-47d5-4b13-af8c-596b8dc03d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:03.663011 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1388/2761] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:03.668237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d16fd1-67d5-43ae-b164-b027152f1bb1 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.729000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d16fd1-67d5-43ae-b164-b027152f1bb1 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.733588 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af61bb9-b1e6-4b0f-adb1-bb3f45a5eaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.734187 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af61bb9-b1e6-4b0f-adb1-bb3f45a5eaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.734482 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af61bb9-b1e6-4b0f-adb1-bb3f45a5eaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.734729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af61bb9-b1e6-4b0f-adb1-bb3f45a5eaa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.735342 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af61bb9-b1e6-4b0f-adb1-bb3f45a5eaa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.737206 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af61bb9-b1e6-4b0f-adb1-bb3f45a5eaa8 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.762890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1374/2762] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:03 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:03.767017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43133e88-47d5-4b13-af8c-596b8dc03d2b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.777743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43133e88-47d5-4b13-af8c-596b8dc03d2b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.845747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087b7d4-d86c-4f70-9eef-2929f00700fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.846260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087b7d4-d86c-4f70-9eef-2929f00700fe admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.846537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087b7d4-d86c-4f70-9eef-2929f00700fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.847559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087b7d4-d86c-4f70-9eef-2929f00700fe admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.848189 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087b7d4-d86c-4f70-9eef-2929f00700fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.850126 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087b7d4-d86c-4f70-9eef-2929f00700fe admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.878954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1389/2763] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:03 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:03.880247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af61bb9-b1e6-4b0f-adb1-bb3f45a5eaa8 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.890760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af61bb9-b1e6-4b0f-adb1-bb3f45a5eaa8 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.895900 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee772df5-7390-478a-a45d-dd88406d162e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.896495 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee772df5-7390-478a-a45d-dd88406d162e admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.896796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee772df5-7390-478a-a45d-dd88406d162e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.897090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee772df5-7390-478a-a45d-dd88406d162e admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.897760 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee772df5-7390-478a-a45d-dd88406d162e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.899334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee772df5-7390-478a-a45d-dd88406d162e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.910252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1375/2764] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:30:03.914516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087b7d4-d86c-4f70-9eef-2929f00700fe admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.924420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087b7d4-d86c-4f70-9eef-2929f00700fe admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.929210 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd992f8-fc65-43ab-b62d-bc69a423914a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:03.929890 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd992f8-fc65-43ab-b62d-bc69a423914a admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:03.930165 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd992f8-fc65-43ab-b62d-bc69a423914a admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:03.930456 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd992f8-fc65-43ab-b62d-bc69a423914a admin admin] PATH_INFO: `/v3/projects/4eb49f653f8849609976d9689aa920dd/users/7d1a2576f9f04d8bb3efc05b0d4a0c8c/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:03.945195 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd992f8-fc65-43ab-b62d-bc69a423914a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb49f653f8849609976d9689aa920dd, user_id=7d1a2576f9f04d8bb3efc05b0d4a0c8c, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:03.947939 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd992f8-fc65-43ab-b62d-bc69a423914a admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:03.960553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1390/2765] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:03 2022] PUT /identity/v3/projects/4eb49f653f8849609976d9689aa920dd/users/7d1a2576f9f04d8bb3efc05b0d4a0c8c/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:03.964330 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee772df5-7390-478a-a45d-dd88406d162e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:03.974843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee772df5-7390-478a-a45d-dd88406d162e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:03.978362 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:04.031796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4d848-eea3-48f2-94e5-15c8931dab06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:04.032392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4d848-eea3-48f2-94e5-15c8931dab06 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:04.032667 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4d848-eea3-48f2-94e5-15c8931dab06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:04.032932 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4d848-eea3-48f2-94e5-15c8931dab06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:04.033584 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4d848-eea3-48f2-94e5-15c8931dab06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:04.035754 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4d848-eea3-48f2-94e5-15c8931dab06 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:04.045184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1376/2766] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:04.049620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd992f8-fc65-43ab-b62d-bc69a423914a admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:04.062541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd992f8-fc65-43ab-b62d-bc69a423914a admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:04.067453 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ee9fa-c366-49ce-9c7c-0d557892c9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:04.068115 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ee9fa-c366-49ce-9c7c-0d557892c9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:04.068395 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ee9fa-c366-49ce-9c7c-0d557892c9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:04.068653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ee9fa-c366-49ce-9c7c-0d557892c9a9 admin admin] PATH_INFO: `/v3/projects/4eb49f653f8849609976d9689aa920dd/users/7d1a2576f9f04d8bb3efc05b0d4a0c8c/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:04.070589 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ee9fa-c366-49ce-9c7c-0d557892c9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb49f653f8849609976d9689aa920dd, user_id=7d1a2576f9f04d8bb3efc05b0d4a0c8c, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:04.073339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ee9fa-c366-49ce-9c7c-0d557892c9a9 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:04.084393 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1391/2767] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:04 2022] PUT /identity/v3/projects/4eb49f653f8849609976d9689aa920dd/users/7d1a2576f9f04d8bb3efc05b0d4a0c8c/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:04.094680 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4c1aa-c575-45a8-a493-af94eb8093fe None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:04.094961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4c1aa-c575-45a8-a493-af94eb8093fe None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:04.095229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4c1aa-c575-45a8-a493-af94eb8093fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:04.127941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-3bb4c1aa-c575-45a8-a493-af94eb8093fe None None] MFA Rules not processed for user `7d1a2576f9f04d8bb3efc05b0d4a0c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:04.129493 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3bb4c1aa-c575-45a8-a493-af94eb8093fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:04.201736 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1377/2768] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:04.213294 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ee9fa-c366-49ce-9c7c-0d557892c9a9 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:04.215012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-902ee9fa-c366-49ce-9c7c-0d557892c9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:04.239035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:04.276072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ee9fa-c366-49ce-9c7c-0d557892c9a9 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:04.279710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a9f94-9be3-49bd-84ef-fb9a54b9b1a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:04.280305 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a9f94-9be3-49bd-84ef-fb9a54b9b1a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:04.280583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a9f94-9be3-49bd-84ef-fb9a54b9b1a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:04.280849 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a9f94-9be3-49bd-84ef-fb9a54b9b1a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:04.289531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a9f94-9be3-49bd-84ef-fb9a54b9b1a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:04.290967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a9f94-9be3-49bd-84ef-fb9a54b9b1a1 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:04.300301 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1392/2769] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:04.373933 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4c1aa-c575-45a8-a493-af94eb8093fe None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:04.384428 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4c1aa-c575-45a8-a493-af94eb8093fe None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:04.388039 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42627f0a-69ad-4204-924c-6a71b9f99d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:04.388635 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42627f0a-69ad-4204-924c-6a71b9f99d40 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:04.388912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42627f0a-69ad-4204-924c-6a71b9f99d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:04.389183 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42627f0a-69ad-4204-924c-6a71b9f99d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:04.454683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42627f0a-69ad-4204-924c-6a71b9f99d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:04.456111 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42627f0a-69ad-4204-924c-6a71b9f99d40 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:04.465734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1378/2770] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:07.213873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a9f94-9be3-49bd-84ef-fb9a54b9b1a1 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:07.275985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a9f94-9be3-49bd-84ef-fb9a54b9b1a1 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:07.280652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:07.281299 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:07.281688 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:07.281986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] PATH_INFO: `/v3/users/784549310ee74732838da7d18d82cc53` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:07.283127 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=784549310ee74732838da7d18d82cc53)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:07.285041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:07.332315 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784549310ee74732838da7d18d82cc53'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.332970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784549310ee74732838da7d18d82cc53'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.339714 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784549310ee74732838da7d18d82cc53'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.340327 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784549310ee74732838da7d18d82cc53'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.347608 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784549310ee74732838da7d18d82cc53'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.364491 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1393/2771] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:07 2022] DELETE /identity/v3/users/784549310ee74732838da7d18d82cc53 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:07.368914 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa348-3e79-485c-811a-eb7e07d1fcf7 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:07.369374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa348-3e79-485c-811a-eb7e07d1fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:07.369496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fa348-3e79-485c-811a-eb7e07d1fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:07.388472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-3d3fa348-3e79-485c-811a-eb7e07d1fcf7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:07.389694 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3d3fa348-3e79-485c-811a-eb7e07d1fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:07.412556 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:07.452401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1379/2772] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:07.463270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:07.464710 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:07.509976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:07.523718 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083f5a1-0ff1-47c1-bcba-354877e71f4e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:07.527402 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52774d2-5935-4c65-a881-80bd73c88df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:07.528005 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52774d2-5935-4c65-a881-80bd73c88df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:07.528284 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52774d2-5935-4c65-a881-80bd73c88df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:07.528552 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52774d2-5935-4c65-a881-80bd73c88df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:07.537915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52774d2-5935-4c65-a881-80bd73c88df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:07.539338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52774d2-5935-4c65-a881-80bd73c88df0 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:07.548623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1394/2773] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:07.887590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fa348-3e79-485c-811a-eb7e07d1fcf7 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:07.888994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3d3fa348-3e79-485c-811a-eb7e07d1fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:07.901498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fa348-3e79-485c-811a-eb7e07d1fcf7 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:07.906281 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:07.906911 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:07.907186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:07.907456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] PATH_INFO: `/v3/projects/9ae14f049fd645b6b7ce36192ab165d6` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:07.908605 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ae14f049fd645b6b7ce36192ab165d6)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:07.910545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:07.958150 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ae14f049fd645b6b7ce36192ab165d6', 'request_id': 'req-f9dfecf9-9e20-4344-977f-778676f976f5'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.965472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ae14f049fd645b6b7ce36192ab165d6', 'request_id': 'req-f9dfecf9-9e20-4344-977f-778676f976f5'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.971805 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ae14f049fd645b6b7ce36192ab165d6', 'request_id': 'req-f9dfecf9-9e20-4344-977f-778676f976f5'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.982972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] The token cache is being invalidate because project 9ae14f049fd645b6b7ce36192ab165d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:07.983257 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.983880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:07.985331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1380/2774] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:07 2022] DELETE /identity/v3/projects/9ae14f049fd645b6b7ce36192ab165d6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:07.989839 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52774d2-5935-4c65-a881-80bd73c88df0 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:07.999277 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52774d2-5935-4c65-a881-80bd73c88df0 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:08.061912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:08.062478 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:08.062977 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:08.063250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] PATH_INFO: `/v3/users/7d1a2576f9f04d8bb3efc05b0d4a0c8c` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:08.064586 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1a2576f9f04d8bb3efc05b0d4a0c8c)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:08.066894 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:08.126623 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1a2576f9f04d8bb3efc05b0d4a0c8c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.127389 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1a2576f9f04d8bb3efc05b0d4a0c8c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.135407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1a2576f9f04d8bb3efc05b0d4a0c8c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.136087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1a2576f9f04d8bb3efc05b0d4a0c8c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.143630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-40a91e46-8b87-4147-a29c-37ba4f973b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1a2576f9f04d8bb3efc05b0d4a0c8c'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.162021 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1395/2775] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:07 2022] DELETE /identity/v3/users/7d1a2576f9f04d8bb3efc05b0d4a0c8c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:08.444986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:08.448807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:08.466041 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfecf9-9e20-4344-977f-778676f976f5 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:08.524400 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:08.525025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:08.525300 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:08.525603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] PATH_INFO: `/v3/projects/4eb49f653f8849609976d9689aa920dd` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:08.526700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eb49f653f8849609976d9689aa920dd)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:08.528626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:08.574682 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eb49f653f8849609976d9689aa920dd', 'request_id': 'req-e6c1275c-2adb-4e14-be18-3930e2448032'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.577843 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:08.584788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eb49f653f8849609976d9689aa920dd', 'request_id': 'req-e6c1275c-2adb-4e14-be18-3930e2448032'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.597809 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eb49f653f8849609976d9689aa920dd', 'request_id': 'req-e6c1275c-2adb-4e14-be18-3930e2448032'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.608171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] The token cache is being invalidate because project 4eb49f653f8849609976d9689aa920dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:08.608477 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.609152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e6c1275c-2adb-4e14-be18-3930e2448032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:08.610743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1381/2776] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:08 2022] DELETE /identity/v3/projects/4eb49f653f8849609976d9689aa920dd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:30:08.616421 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2edcdc-83cd-4e22-9273-56ef1e75ccf2 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:08.616945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2edcdc-83cd-4e22-9273-56ef1e75ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:08.617044 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2edcdc-83cd-4e22-9273-56ef1e75ccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:08.639597 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-1a2edcdc-83cd-4e22-9273-56ef1e75ccf2 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:08.640806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1a2edcdc-83cd-4e22-9273-56ef1e75ccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:08.698826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1396/2777] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:08.707601 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5bec-aa0c-4ab5-8ce0-f5e280fda42b None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:08.707884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5bec-aa0c-4ab5-8ce0-f5e280fda42b None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:08.708182 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5bec-aa0c-4ab5-8ce0-f5e280fda42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:08.728591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-1beb5bec-aa0c-4ab5-8ce0-f5e280fda42b None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:08.729946 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-1beb5bec-aa0c-4ab5-8ce0-f5e280fda42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:08.738746 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1382/2778] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:08.742993 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2edcdc-83cd-4e22-9273-56ef1e75ccf2 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:08.749255 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:08.753479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2edcdc-83cd-4e22-9273-56ef1e75ccf2 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:08.759224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af7f0-f0e9-4b91-aebf-8043c145d8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:08.759842 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af7f0-f0e9-4b91-aebf-8043c145d8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:08.760460 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af7f0-f0e9-4b91-aebf-8043c145d8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:08.760995 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354af7f0-f0e9-4b91-aebf-8043c145d8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:08.761789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af7f0-f0e9-4b91-aebf-8043c145d8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:30:08.762295 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af7f0-f0e9-4b91-aebf-8043c145d8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:08.763889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354af7f0-f0e9-4b91-aebf-8043c145d8d1 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:08.778968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1397/2779] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:30:08.783709 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb5bec-aa0c-4ab5-8ce0-f5e280fda42b None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:08.794240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb5bec-aa0c-4ab5-8ce0-f5e280fda42b None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:08.802649 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81b17c-2e4d-4ce6-98f0-91727f338faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:08.803685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81b17c-2e4d-4ce6-98f0-91727f338faf admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:08.803959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81b17c-2e4d-4ce6-98f0-91727f338faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:08.804224 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81b17c-2e4d-4ce6-98f0-91727f338faf admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:08.804903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81b17c-2e4d-4ce6-98f0-91727f338faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:08.807202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81b17c-2e4d-4ce6-98f0-91727f338faf admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:08.840777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1383/2780] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:08 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:08.842148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af7f0-f0e9-4b91-aebf-8043c145d8d1 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:08.852727 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354af7f0-f0e9-4b91-aebf-8043c145d8d1 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:08.910415 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4478d-497f-4fb8-97bd-9a8daf41b05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:08.911328 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4478d-497f-4fb8-97bd-9a8daf41b05f admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:08.911599 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4478d-497f-4fb8-97bd-9a8daf41b05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:08.911864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4478d-497f-4fb8-97bd-9a8daf41b05f admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:08.912548 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4478d-497f-4fb8-97bd-9a8daf41b05f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:08.914799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4478d-497f-4fb8-97bd-9a8daf41b05f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:08.935573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1398/2781] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:08 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:08.939664 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81b17c-2e4d-4ce6-98f0-91727f338faf admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:08.950454 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81b17c-2e4d-4ce6-98f0-91727f338faf admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:08.955786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a432eb-64fd-431b-a8bf-3707bce11633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:08.956371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a432eb-64fd-431b-a8bf-3707bce11633 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:08.956783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a432eb-64fd-431b-a8bf-3707bce11633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:08.957057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a432eb-64fd-431b-a8bf-3707bce11633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:08.957727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a432eb-64fd-431b-a8bf-3707bce11633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:08.959302 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a432eb-64fd-431b-a8bf-3707bce11633 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:08.969027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1384/2782] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:08.973333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4478d-497f-4fb8-97bd-9a8daf41b05f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:08.983079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4478d-497f-4fb8-97bd-9a8daf41b05f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:08.987569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac79f31-f7f8-4d26-bf20-b4aac5163997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:08.988825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac79f31-f7f8-4d26-bf20-b4aac5163997 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:08.989096 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac79f31-f7f8-4d26-bf20-b4aac5163997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:08.989362 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac79f31-f7f8-4d26-bf20-b4aac5163997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:08.990018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac79f31-f7f8-4d26-bf20-b4aac5163997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:08.991579 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac79f31-f7f8-4d26-bf20-b4aac5163997 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.002276 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1399/2783] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:09.006190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a432eb-64fd-431b-a8bf-3707bce11633 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:09.016128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a432eb-64fd-431b-a8bf-3707bce11633 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:09.020695 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6641c65-2585-4aff-bfbe-ee0d00cd7c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:09.021334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6641c65-2585-4aff-bfbe-ee0d00cd7c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.021652 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6641c65-2585-4aff-bfbe-ee0d00cd7c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.021926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6641c65-2585-4aff-bfbe-ee0d00cd7c6b admin admin] PATH_INFO: `/v3/projects/ceb2eb84e7c444c4bc94b0232b7590b5/users/b92f88fc9d7643e59bfc77f50cf8ada3/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.039160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6641c65-2585-4aff-bfbe-ee0d00cd7c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb2eb84e7c444c4bc94b0232b7590b5, user_id=b92f88fc9d7643e59bfc77f50cf8ada3, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:09.042252 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6641c65-2585-4aff-bfbe-ee0d00cd7c6b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.058954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1385/2784] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:09 2022] PUT /identity/v3/projects/ceb2eb84e7c444c4bc94b0232b7590b5/users/b92f88fc9d7643e59bfc77f50cf8ada3/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:09.069444 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74eca05-9636-468f-9b9b-c0f91802f29f None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.069774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74eca05-9636-468f-9b9b-c0f91802f29f None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.070064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74eca05-9636-468f-9b9b-c0f91802f29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.097945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b74eca05-9636-468f-9b9b-c0f91802f29f None None] MFA Rules not processed for user `b92f88fc9d7643e59bfc77f50cf8ada3`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:09.099164 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b74eca05-9636-468f-9b9b-c0f91802f29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:09.162270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1400/2785] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:09.166724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6641c65-2585-4aff-bfbe-ee0d00cd7c6b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:09.227057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6641c65-2585-4aff-bfbe-ee0d00cd7c6b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:09.231615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50a905-8fbe-420e-bb92-e96cccd36aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:09.232213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50a905-8fbe-420e-bb92-e96cccd36aba admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.232491 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50a905-8fbe-420e-bb92-e96cccd36aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.232759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50a905-8fbe-420e-bb92-e96cccd36aba admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.233372 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50a905-8fbe-420e-bb92-e96cccd36aba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:09.235290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50a905-8fbe-420e-bb92-e96cccd36aba admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.262627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1386/2786] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:09 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:09.266673 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74eca05-9636-468f-9b9b-c0f91802f29f None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:09.278132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74eca05-9636-468f-9b9b-c0f91802f29f None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:09.343937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d5a8c-92cb-404f-b7dd-9800f3e761fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:09.344546 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d5a8c-92cb-404f-b7dd-9800f3e761fe admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.344822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d5a8c-92cb-404f-b7dd-9800f3e761fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.345091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d5a8c-92cb-404f-b7dd-9800f3e761fe admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.345744 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d5a8c-92cb-404f-b7dd-9800f3e761fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:09.347654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d5a8c-92cb-404f-b7dd-9800f3e761fe admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.355516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:09.369668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1401/2787] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:09 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:09.373696 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50a905-8fbe-420e-bb92-e96cccd36aba admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:09.384379 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50a905-8fbe-420e-bb92-e96cccd36aba admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:09.388853 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d9d83-6e0d-4c41-a7cc-0d61061b9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:09.389436 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d9d83-6e0d-4c41-a7cc-0d61061b9e0f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.389745 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d9d83-6e0d-4c41-a7cc-0d61061b9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.390011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d9d83-6e0d-4c41-a7cc-0d61061b9e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.390645 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d9d83-6e0d-4c41-a7cc-0d61061b9e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:09.392212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d9d83-6e0d-4c41-a7cc-0d61061b9e0f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.405630 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1387/2788] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:09.409742 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d5a8c-92cb-404f-b7dd-9800f3e761fe admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:09.419725 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d5a8c-92cb-404f-b7dd-9800f3e761fe admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:09.424632 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81691d7d-d9a1-4b3f-875d-c3829e8d39e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:09.425815 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81691d7d-d9a1-4b3f-875d-c3829e8d39e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.426089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81691d7d-d9a1-4b3f-875d-c3829e8d39e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.426441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81691d7d-d9a1-4b3f-875d-c3829e8d39e6 admin admin] PATH_INFO: `/v3/projects/8e0aabc5e8b24076a7e6f052239a1815/users/5d4d6224eb4f49f68ed35444eccfc17d/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.443876 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81691d7d-d9a1-4b3f-875d-c3829e8d39e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0aabc5e8b24076a7e6f052239a1815, user_id=5d4d6224eb4f49f68ed35444eccfc17d, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:09.446633 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81691d7d-d9a1-4b3f-875d-c3829e8d39e6 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.460424 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1402/2789] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:09 2022] PUT /identity/v3/projects/8e0aabc5e8b24076a7e6f052239a1815/users/5d4d6224eb4f49f68ed35444eccfc17d/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:09.464669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d9d83-6e0d-4c41-a7cc-0d61061b9e0f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:09.475626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d9d83-6e0d-4c41-a7cc-0d61061b9e0f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:09.520781 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:09.531863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e105e-e9c7-4d57-a397-c46fb3a25ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:09.532457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e105e-e9c7-4d57-a397-c46fb3a25ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.532731 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e105e-e9c7-4d57-a397-c46fb3a25ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.532996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e105e-e9c7-4d57-a397-c46fb3a25ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.533748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e105e-e9c7-4d57-a397-c46fb3a25ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:09.535856 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e105e-e9c7-4d57-a397-c46fb3a25ca2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.545597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1388/2790] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:09.549649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81691d7d-d9a1-4b3f-875d-c3829e8d39e6 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:09.560647 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81691d7d-d9a1-4b3f-875d-c3829e8d39e6 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:09.565206 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b1762-f86c-4e70-81b9-c182d84dc9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:09.565883 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b1762-f86c-4e70-81b9-c182d84dc9a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.566162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b1762-f86c-4e70-81b9-c182d84dc9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.566435 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b1762-f86c-4e70-81b9-c182d84dc9a5 admin admin] PATH_INFO: `/v3/projects/8e0aabc5e8b24076a7e6f052239a1815/users/5d4d6224eb4f49f68ed35444eccfc17d/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.568602 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b1762-f86c-4e70-81b9-c182d84dc9a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0aabc5e8b24076a7e6f052239a1815, user_id=5d4d6224eb4f49f68ed35444eccfc17d, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:09.571280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b1762-f86c-4e70-81b9-c182d84dc9a5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.581483 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1403/2791] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:09 2022] PUT /identity/v3/projects/8e0aabc5e8b24076a7e6f052239a1815/users/5d4d6224eb4f49f68ed35444eccfc17d/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:09.591640 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854c05c-3ba8-4015-a0ac-507d22353d5c None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.591920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854c05c-3ba8-4015-a0ac-507d22353d5c None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.592190 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854c05c-3ba8-4015-a0ac-507d22353d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.619743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8854c05c-3ba8-4015-a0ac-507d22353d5c None None] MFA Rules not processed for user `5d4d6224eb4f49f68ed35444eccfc17d`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:09.620925 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8854c05c-3ba8-4015-a0ac-507d22353d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:09.693530 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1389/2792] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:09.704972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b1762-f86c-4e70-81b9-c182d84dc9a5 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:09.706961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-310b1762-f86c-4e70-81b9-c182d84dc9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:09.766651 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b1762-f86c-4e70-81b9-c182d84dc9a5 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:09.771168 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8b586-9f60-43fe-b307-bf97b8d0c762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:09.771871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8b586-9f60-43fe-b307-bf97b8d0c762 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.772149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8b586-9f60-43fe-b307-bf97b8d0c762 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.772448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8b586-9f60-43fe-b307-bf97b8d0c762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.785848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8b586-9f60-43fe-b307-bf97b8d0c762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:09.787840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8b586-9f60-43fe-b307-bf97b8d0c762 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.797720 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1404/2793] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:09.848576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854c05c-3ba8-4015-a0ac-507d22353d5c None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:09.850011 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8854c05c-3ba8-4015-a0ac-507d22353d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:09.862485 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854c05c-3ba8-4015-a0ac-507d22353d5c None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:09.868415 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a2ada-6a6b-4d23-84ae-62bbe9f3f53d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:09.869028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a2ada-6a6b-4d23-84ae-62bbe9f3f53d service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:09.869303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a2ada-6a6b-4d23-84ae-62bbe9f3f53d service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:09.869602 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a2ada-6a6b-4d23-84ae-62bbe9f3f53d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:09.929788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a2ada-6a6b-4d23-84ae-62bbe9f3f53d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:09.931205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a2ada-6a6b-4d23-84ae-62bbe9f3f53d service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:09.939876 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1390/2794] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:13.328380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8b586-9f60-43fe-b307-bf97b8d0c762 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:13.392082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8b586-9f60-43fe-b307-bf97b8d0c762 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:13.397781 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:13.398617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:13.398934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:13.399288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] PATH_INFO: `/v3/users/b92f88fc9d7643e59bfc77f50cf8ada3` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:13.400854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b92f88fc9d7643e59bfc77f50cf8ada3)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:13.403449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:13.456323 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b92f88fc9d7643e59bfc77f50cf8ada3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:13.457007 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b92f88fc9d7643e59bfc77f50cf8ada3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:13.464041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b92f88fc9d7643e59bfc77f50cf8ada3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:13.464712 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b92f88fc9d7643e59bfc77f50cf8ada3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:13.466410 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:13.477176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b92f88fc9d7643e59bfc77f50cf8ada3'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:13.499181 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1405/2795] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:13 2022] DELETE /identity/v3/users/b92f88fc9d7643e59bfc77f50cf8ada3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:13.503854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa160601-dd36-4ffb-9d25-8db4adde6f75 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:13.504660 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa160601-dd36-4ffb-9d25-8db4adde6f75 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:13.504830 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa160601-dd36-4ffb-9d25-8db4adde6f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:13.525687 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-fa160601-dd36-4ffb-9d25-8db4adde6f75 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:13.526863 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fa160601-dd36-4ffb-9d25-8db4adde6f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:13.592799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1391/2796] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:13.603581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:13.605082 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:13.685250 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e2cd2-4cf2-479f-af0d-b2322041bc50 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:13.689033 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2ca80-35e3-4e84-b5ea-08289b3d936c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:13.689670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2ca80-35e3-4e84-b5ea-08289b3d936c service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:13.689953 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2ca80-35e3-4e84-b5ea-08289b3d936c service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:13.690224 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2ca80-35e3-4e84-b5ea-08289b3d936c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:13.702200 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2ca80-35e3-4e84-b5ea-08289b3d936c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:13.703661 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2ca80-35e3-4e84-b5ea-08289b3d936c service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:13.714035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1406/2797] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:14.001102 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa160601-dd36-4ffb-9d25-8db4adde6f75 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:14.002866 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-fa160601-dd36-4ffb-9d25-8db4adde6f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:14.015424 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa160601-dd36-4ffb-9d25-8db4adde6f75 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:14.026847 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:14.028465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:14.028980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:14.029349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] PATH_INFO: `/v3/projects/ceb2eb84e7c444c4bc94b0232b7590b5` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:14.033969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb2eb84e7c444c4bc94b0232b7590b5)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:14.039080 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:14.058568 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:14.109201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb2eb84e7c444c4bc94b0232b7590b5', 'request_id': 'req-187c5eae-62ae-4dfb-99a0-ac611d3ed323'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.119671 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb2eb84e7c444c4bc94b0232b7590b5', 'request_id': 'req-187c5eae-62ae-4dfb-99a0-ac611d3ed323'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.126654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb2eb84e7c444c4bc94b0232b7590b5', 'request_id': 'req-187c5eae-62ae-4dfb-99a0-ac611d3ed323'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.139884 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] The token cache is being invalidate because project ceb2eb84e7c444c4bc94b0232b7590b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:14.140177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.140831 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.142338 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1392/2798] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:14 2022] DELETE /identity/v3/projects/ceb2eb84e7c444c4bc94b0232b7590b5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:14.157640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2ca80-35e3-4e84-b5ea-08289b3d936c service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:14.167935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2ca80-35e3-4e84-b5ea-08289b3d936c service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:14.246884 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:14.247796 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:14.248157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:14.248535 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] PATH_INFO: `/v3/users/5d4d6224eb4f49f68ed35444eccfc17d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:14.257023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d4d6224eb4f49f68ed35444eccfc17d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:14.259554 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:14.319571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d4d6224eb4f49f68ed35444eccfc17d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.320236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d4d6224eb4f49f68ed35444eccfc17d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.327852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d4d6224eb4f49f68ed35444eccfc17d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.328498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d4d6224eb4f49f68ed35444eccfc17d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.331064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:14.337873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-e728dee4-f9af-4d71-914a-9ca5bd835e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d4d6224eb4f49f68ed35444eccfc17d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.362132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1407/2799] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:14 2022] DELETE /identity/v3/users/5d4d6224eb4f49f68ed35444eccfc17d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:14.671100 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:14.675266 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:14.695326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c5eae-62ae-4dfb-99a0-ac611d3ed323 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:14.755142 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:14.755777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:14.756056 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:14.756328 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] PATH_INFO: `/v3/projects/8e0aabc5e8b24076a7e6f052239a1815` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:14.757434 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e0aabc5e8b24076a7e6f052239a1815)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:14.759354 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:14.808249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e0aabc5e8b24076a7e6f052239a1815', 'request_id': 'req-e869e557-f979-47fb-8864-5f6747c16098'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.817979 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e0aabc5e8b24076a7e6f052239a1815', 'request_id': 'req-e869e557-f979-47fb-8864-5f6747c16098'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.825128 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e0aabc5e8b24076a7e6f052239a1815', 'request_id': 'req-e869e557-f979-47fb-8864-5f6747c16098'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.838573 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] The token cache is being invalidate because project 8e0aabc5e8b24076a7e6f052239a1815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:14.838880 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.839525 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e869e557-f979-47fb-8864-5f6747c16098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:14.841228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1393/2800] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:14 2022] DELETE /identity/v3/projects/8e0aabc5e8b24076a7e6f052239a1815 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:14.846374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d5c79-c1ab-41b4-bba5-f9b2913ef8be None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:14.846780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d5c79-c1ab-41b4-bba5-f9b2913ef8be None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:14.847000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d5c79-c1ab-41b4-bba5-f9b2913ef8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:14.872376 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-fa2d5c79-c1ab-41b4-bba5-f9b2913ef8be None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:14.873583 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-fa2d5c79-c1ab-41b4-bba5-f9b2913ef8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:14.946987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1408/2801] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:14.955236 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bae92-1e4d-487c-8028-15030749f84f None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:14.955629 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bae92-1e4d-487c-8028-15030749f84f None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:14.955944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bae92-1e4d-487c-8028-15030749f84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:14.976964 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-5c4bae92-1e4d-487c-8028-15030749f84f None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:14.978198 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5c4bae92-1e4d-487c-8028-15030749f84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:14.987986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1394/2802] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:15.002116 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d5c79-c1ab-41b4-bba5-f9b2913ef8be None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.030198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d5c79-c1ab-41b4-bba5-f9b2913ef8be None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.035153 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a82c31-9214-442c-888f-63ef7103de2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.035863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a82c31-9214-442c-888f-63ef7103de2e admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.036035 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a82c31-9214-442c-888f-63ef7103de2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.036565 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a82c31-9214-442c-888f-63ef7103de2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.037289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a82c31-9214-442c-888f-63ef7103de2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:30:15.037474 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a82c31-9214-442c-888f-63ef7103de2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.039462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a82c31-9214-442c-888f-63ef7103de2e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.058606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bae92-1e4d-487c-8028-15030749f84f None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.061851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1409/2803] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:30:15.070587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bae92-1e4d-487c-8028-15030749f84f None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.076452 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5706a6c-c230-4395-a8aa-10bb71348c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.085823 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5706a6c-c230-4395-a8aa-10bb71348c2b admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.088623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5706a6c-c230-4395-a8aa-10bb71348c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.088623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5706a6c-c230-4395-a8aa-10bb71348c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.088623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5706a6c-c230-4395-a8aa-10bb71348c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.096747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5706a6c-c230-4395-a8aa-10bb71348c2b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.115334 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:15.140924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1395/2804] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:15 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:15.144952 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a82c31-9214-442c-888f-63ef7103de2e admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.155506 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a82c31-9214-442c-888f-63ef7103de2e admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.220584 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ce32f-755a-40b0-b226-9c1a995ef1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.221454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ce32f-755a-40b0-b226-9c1a995ef1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.221850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ce32f-755a-40b0-b226-9c1a995ef1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.222428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ce32f-755a-40b0-b226-9c1a995ef1b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.222990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ce32f-755a-40b0-b226-9c1a995ef1b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.225041 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ce32f-755a-40b0-b226-9c1a995ef1b7 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.248357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1410/2805] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:15 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:15.252374 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5706a6c-c230-4395-a8aa-10bb71348c2b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.263315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5706a6c-c230-4395-a8aa-10bb71348c2b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.268636 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a3786-72be-4f30-84b8-b1cac6b15a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.269228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a3786-72be-4f30-84b8-b1cac6b15a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.269504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a3786-72be-4f30-84b8-b1cac6b15a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.269832 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a3786-72be-4f30-84b8-b1cac6b15a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.270462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a3786-72be-4f30-84b8-b1cac6b15a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.272070 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a3786-72be-4f30-84b8-b1cac6b15a0e admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.283229 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1396/2806] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:15.287614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ce32f-755a-40b0-b226-9c1a995ef1b7 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.298462 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ce32f-755a-40b0-b226-9c1a995ef1b7 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.303626 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fdff2-a0a7-4171-badb-a0676d4c295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.304217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fdff2-a0a7-4171-badb-a0676d4c295b admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.304492 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fdff2-a0a7-4171-badb-a0676d4c295b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.304760 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fdff2-a0a7-4171-badb-a0676d4c295b admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.305403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0fdff2-a0a7-4171-badb-a0676d4c295b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.307020 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0fdff2-a0a7-4171-badb-a0676d4c295b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.318617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1411/2807] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:30:15.323053 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a3786-72be-4f30-84b8-b1cac6b15a0e admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.334615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a3786-72be-4f30-84b8-b1cac6b15a0e admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.339347 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1edc0-d215-4b8f-9a06-d9712da8729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.339996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1edc0-d215-4b8f-9a06-d9712da8729d admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.340274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1edc0-d215-4b8f-9a06-d9712da8729d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.340548 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1edc0-d215-4b8f-9a06-d9712da8729d admin admin] PATH_INFO: `/v3/projects/cd315fc7e8cf415e84a546002524ea4a/users/01b1c18e91b642a9aaafd1c24e33f3e7/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.355460 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1edc0-d215-4b8f-9a06-d9712da8729d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd315fc7e8cf415e84a546002524ea4a, user_id=01b1c18e91b642a9aaafd1c24e33f3e7, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.358315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1edc0-d215-4b8f-9a06-d9712da8729d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.372778 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1397/2808] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:15 2022] PUT /identity/v3/projects/cd315fc7e8cf415e84a546002524ea4a/users/01b1c18e91b642a9aaafd1c24e33f3e7/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:15.383537 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c13b34-186f-4f4f-a8a0-a739979d2c1a None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.384262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c13b34-186f-4f4f-a8a0-a739979d2c1a None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.384368 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c13b34-186f-4f4f-a8a0-a739979d2c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.412775 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-b4c13b34-186f-4f4f-a8a0-a739979d2c1a None None] MFA Rules not processed for user `01b1c18e91b642a9aaafd1c24e33f3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:15.414137 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b4c13b34-186f-4f4f-a8a0-a739979d2c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:15.447785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:15.487022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1412/2809] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:15.492314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1edc0-d215-4b8f-9a06-d9712da8729d admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.563575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1edc0-d215-4b8f-9a06-d9712da8729d admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.568391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7350d1b-728e-473c-86d6-eaed84a4ab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.568994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7350d1b-728e-473c-86d6-eaed84a4ab52 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.569272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7350d1b-728e-473c-86d6-eaed84a4ab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.569590 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7350d1b-728e-473c-86d6-eaed84a4ab52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.570225 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7350d1b-728e-473c-86d6-eaed84a4ab52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.572104 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7350d1b-728e-473c-86d6-eaed84a4ab52 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.602779 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1398/2810] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:15 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:15.604219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c13b34-186f-4f4f-a8a0-a739979d2c1a None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.616756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c13b34-186f-4f4f-a8a0-a739979d2c1a None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.675780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec1e6-6958-4bb7-bab5-86770d6e3c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.676958 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec1e6-6958-4bb7-bab5-86770d6e3c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.677231 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec1e6-6958-4bb7-bab5-86770d6e3c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.677495 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ec1e6-6958-4bb7-bab5-86770d6e3c01 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.678144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec1e6-6958-4bb7-bab5-86770d6e3c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.680051 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ec1e6-6958-4bb7-bab5-86770d6e3c01 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.703274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1413/2811] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:15 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:15.707441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7350d1b-728e-473c-86d6-eaed84a4ab52 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.717699 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7350d1b-728e-473c-86d6-eaed84a4ab52 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.723545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f4ae6-aaba-4a2a-90b2-d7bed24523fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.724146 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f4ae6-aaba-4a2a-90b2-d7bed24523fb admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.724419 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f4ae6-aaba-4a2a-90b2-d7bed24523fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.724684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f4ae6-aaba-4a2a-90b2-d7bed24523fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.725322 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f4ae6-aaba-4a2a-90b2-d7bed24523fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.726965 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f4ae6-aaba-4a2a-90b2-d7bed24523fb admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.737973 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1399/2812] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:15.742949 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec1e6-6958-4bb7-bab5-86770d6e3c01 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.753012 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ec1e6-6958-4bb7-bab5-86770d6e3c01 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.757748 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f98b8-d7a4-4dd0-a48c-5d11f29e7938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.758340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f98b8-d7a4-4dd0-a48c-5d11f29e7938 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.758618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f98b8-d7a4-4dd0-a48c-5d11f29e7938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.758882 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f98b8-d7a4-4dd0-a48c-5d11f29e7938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.759512 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f98b8-d7a4-4dd0-a48c-5d11f29e7938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.761088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f98b8-d7a4-4dd0-a48c-5d11f29e7938 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.770981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1414/2813] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:15.775164 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f4ae6-aaba-4a2a-90b2-d7bed24523fb admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:15.785154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f4ae6-aaba-4a2a-90b2-d7bed24523fb admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:15.790303 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc160b-c696-4d15-9ceb-b48541368a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:15.790941 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc160b-c696-4d15-9ceb-b48541368a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.791218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc160b-c696-4d15-9ceb-b48541368a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.791489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc160b-c696-4d15-9ceb-b48541368a2a admin admin] PATH_INFO: `/v3/projects/a00c88d1d28b4c0bbf168e1311085b38/users/cf992020c5f843edad12c8e97c238582/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.809029 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc160b-c696-4d15-9ceb-b48541368a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a00c88d1d28b4c0bbf168e1311085b38, user_id=cf992020c5f843edad12c8e97c238582, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:15.811739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc160b-c696-4d15-9ceb-b48541368a2a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:15.824456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1400/2814] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:15 2022] PUT /identity/v3/projects/a00c88d1d28b4c0bbf168e1311085b38/users/cf992020c5f843edad12c8e97c238582/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:30:15.890649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e61ec-35f5-4d8d-841c-34aed07c4f2b None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:15.890929 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e61ec-35f5-4d8d-841c-34aed07c4f2b None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:15.891198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e61ec-35f5-4d8d-841c-34aed07c4f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:15.919377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-3d6e61ec-35f5-4d8d-841c-34aed07c4f2b None None] MFA Rules not processed for user `cf992020c5f843edad12c8e97c238582`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:15.920571 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-3d6e61ec-35f5-4d8d-841c-34aed07c4f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:15.986699 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1415/2815] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:15.991227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc160b-c696-4d15-9ceb-b48541368a2a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:16.053913 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc160b-c696-4d15-9ceb-b48541368a2a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:16.058936 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40672f-c1c2-4ae4-9c90-1a43d5034663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:16.059574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40672f-c1c2-4ae4-9c90-1a43d5034663 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:16.059854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40672f-c1c2-4ae4-9c90-1a43d5034663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:16.060126 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40672f-c1c2-4ae4-9c90-1a43d5034663 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:16.060751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40672f-c1c2-4ae4-9c90-1a43d5034663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:16.062693 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40672f-c1c2-4ae4-9c90-1a43d5034663 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:16.079741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:16.099939 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1401/2816] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:15 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:16.101377 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e61ec-35f5-4d8d-841c-34aed07c4f2b None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:16.113630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e61ec-35f5-4d8d-841c-34aed07c4f2b None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:16.159588 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:16.188420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c220b-d16c-41e1-9b4f-c4d7ae2247ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:16.188987 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c220b-d16c-41e1-9b4f-c4d7ae2247ae admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:16.189162 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c220b-d16c-41e1-9b4f-c4d7ae2247ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:16.189660 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c220b-d16c-41e1-9b4f-c4d7ae2247ae admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:16.190414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c220b-d16c-41e1-9b4f-c4d7ae2247ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:16.193655 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090c220b-d16c-41e1-9b4f-c4d7ae2247ae admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:16.223568 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1416/2817] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:16 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:16.224951 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40672f-c1c2-4ae4-9c90-1a43d5034663 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:16.242849 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40672f-c1c2-4ae4-9c90-1a43d5034663 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:16.247633 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb179e-3f20-461f-a8a1-880fb2ec19f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:16.248242 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb179e-3f20-461f-a8a1-880fb2ec19f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:16.248518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb179e-3f20-461f-a8a1-880fb2ec19f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:16.248783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb179e-3f20-461f-a8a1-880fb2ec19f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:16.249585 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb179e-3f20-461f-a8a1-880fb2ec19f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:16.251293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb179e-3f20-461f-a8a1-880fb2ec19f9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:16.262891 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1402/2818] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:16.266976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c220b-d16c-41e1-9b4f-c4d7ae2247ae admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:16.277448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c220b-d16c-41e1-9b4f-c4d7ae2247ae admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:16.282093 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726d0c8-c631-45e5-9608-91683b5896a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:16.282735 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726d0c8-c631-45e5-9608-91683b5896a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:16.283024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726d0c8-c631-45e5-9608-91683b5896a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:16.283283 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726d0c8-c631-45e5-9608-91683b5896a9 admin admin] PATH_INFO: `/v3/projects/d63a5a94c26d4e85aa50612d280347b4/users/bc7054b22a364d36acd4fe4f76fa11c8/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:16.300365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726d0c8-c631-45e5-9608-91683b5896a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63a5a94c26d4e85aa50612d280347b4, user_id=bc7054b22a364d36acd4fe4f76fa11c8, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:16.303149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726d0c8-c631-45e5-9608-91683b5896a9 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:16.319303 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1417/2819] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:16 2022] PUT /identity/v3/projects/d63a5a94c26d4e85aa50612d280347b4/users/bc7054b22a364d36acd4fe4f76fa11c8/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:16.323331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb179e-3f20-461f-a8a1-880fb2ec19f9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:16.336549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb179e-3f20-461f-a8a1-880fb2ec19f9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:16.400352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6476162-a41e-4748-a47f-4a23f9721b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:16.400984 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6476162-a41e-4748-a47f-4a23f9721b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:16.401264 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6476162-a41e-4748-a47f-4a23f9721b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:16.401565 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6476162-a41e-4748-a47f-4a23f9721b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:16.402209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6476162-a41e-4748-a47f-4a23f9721b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:16.403799 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6476162-a41e-4748-a47f-4a23f9721b3d admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:16.414291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1403/2820] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:16.418524 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726d0c8-c631-45e5-9608-91683b5896a9 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:16.429658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726d0c8-c631-45e5-9608-91683b5896a9 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:16.434443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774b236-9271-4b76-b8f2-9c8e7ac6ee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:16.435102 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774b236-9271-4b76-b8f2-9c8e7ac6ee01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:16.435375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774b236-9271-4b76-b8f2-9c8e7ac6ee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:16.435645 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774b236-9271-4b76-b8f2-9c8e7ac6ee01 admin admin] PATH_INFO: `/v3/projects/d63a5a94c26d4e85aa50612d280347b4/users/bc7054b22a364d36acd4fe4f76fa11c8/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:16.437609 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774b236-9271-4b76-b8f2-9c8e7ac6ee01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63a5a94c26d4e85aa50612d280347b4, user_id=bc7054b22a364d36acd4fe4f76fa11c8, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:16.440221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774b236-9271-4b76-b8f2-9c8e7ac6ee01 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:16.452163 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1418/2821] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:16 2022] PUT /identity/v3/projects/d63a5a94c26d4e85aa50612d280347b4/users/bc7054b22a364d36acd4fe4f76fa11c8/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:16.462811 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b56d5f-e38e-4db7-b7b4-8e8d5d5345be None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:16.463127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b56d5f-e38e-4db7-b7b4-8e8d5d5345be None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:16.463401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b56d5f-e38e-4db7-b7b4-8e8d5d5345be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:16.492648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-67b56d5f-e38e-4db7-b7b4-8e8d5d5345be None None] MFA Rules not processed for user `bc7054b22a364d36acd4fe4f76fa11c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:16.493903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-67b56d5f-e38e-4db7-b7b4-8e8d5d5345be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:16.567824 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1404/2822] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:16 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:16.578812 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774b236-9271-4b76-b8f2-9c8e7ac6ee01 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:16.580521 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c774b236-9271-4b76-b8f2-9c8e7ac6ee01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:16.640160 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774b236-9271-4b76-b8f2-9c8e7ac6ee01 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:16.643829 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02258b-c1db-4192-a36f-542b30b9d2da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:16.644434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02258b-c1db-4192-a36f-542b30b9d2da service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:16.644708 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02258b-c1db-4192-a36f-542b30b9d2da service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:16.644981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02258b-c1db-4192-a36f-542b30b9d2da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:16.654360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02258b-c1db-4192-a36f-542b30b9d2da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:16.655793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02258b-c1db-4192-a36f-542b30b9d2da service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:16.667493 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1419/2823] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:17.323235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b56d5f-e38e-4db7-b7b4-8e8d5d5345be None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:17.325025 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-67b56d5f-e38e-4db7-b7b4-8e8d5d5345be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:17.340551 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b56d5f-e38e-4db7-b7b4-8e8d5d5345be None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:17.344639 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3de11-a7ff-4261-94d5-45bf2953cd93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:17.345249 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3de11-a7ff-4261-94d5-45bf2953cd93 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:17.345533 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3de11-a7ff-4261-94d5-45bf2953cd93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:17.345895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3de11-a7ff-4261-94d5-45bf2953cd93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:17.414214 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3de11-a7ff-4261-94d5-45bf2953cd93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:17.415692 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3de11-a7ff-4261-94d5-45bf2953cd93 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:17.425813 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1405/2824] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:30:24.681497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02258b-c1db-4192-a36f-542b30b9d2da service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:24.699918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02258b-c1db-4192-a36f-542b30b9d2da service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:24.703503 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc096aeb-8fdf-4971-a100-7ccc4d2fe631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:24.704089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc096aeb-8fdf-4971-a100-7ccc4d2fe631 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:24.704369 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc096aeb-8fdf-4971-a100-7ccc4d2fe631 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:24.704629 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc096aeb-8fdf-4971-a100-7ccc4d2fe631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:24.759259 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc096aeb-8fdf-4971-a100-7ccc4d2fe631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:24.760668 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc096aeb-8fdf-4971-a100-7ccc4d2fe631 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:24.769240 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1420/2825] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:26.984987 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3de11-a7ff-4261-94d5-45bf2953cd93 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:26.990895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:27.046336 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3de11-a7ff-4261-94d5-45bf2953cd93 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:27.051577 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:27.052203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:27.052475 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:27.052767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] PATH_INFO: `/v3/users/01b1c18e91b642a9aaafd1c24e33f3e7` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:27.053893 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b1c18e91b642a9aaafd1c24e33f3e7)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:27.055782 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:27.100289 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b1c18e91b642a9aaafd1c24e33f3e7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.100927 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b1c18e91b642a9aaafd1c24e33f3e7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.107839 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b1c18e91b642a9aaafd1c24e33f3e7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.114272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b1c18e91b642a9aaafd1c24e33f3e7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.115097 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b1c18e91b642a9aaafd1c24e33f3e7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.131861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1406/2826] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:26 2022] DELETE /identity/v3/users/01b1c18e91b642a9aaafd1c24e33f3e7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:27.136306 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc20d0-1d47-4a3e-b05a-17204c42c6fa None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:27.136823 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc20d0-1d47-4a3e-b05a-17204c42c6fa None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:27.136934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc20d0-1d47-4a3e-b05a-17204c42c6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:27.156282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-62cc20d0-1d47-4a3e-b05a-17204c42c6fa None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:27.157520 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-62cc20d0-1d47-4a3e-b05a-17204c42c6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:27.176244 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:27.217222 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1421/2827] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:27.227414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:27.228835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:27.286483 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea004d-eaac-4eb7-ba3a-ae3d74f7a82a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:27.290000 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5efc0e-baae-4515-85f0-dd95676c8382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:27.290600 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5efc0e-baae-4515-85f0-dd95676c8382 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:27.290879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5efc0e-baae-4515-85f0-dd95676c8382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:27.291148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5efc0e-baae-4515-85f0-dd95676c8382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:27.300028 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5efc0e-baae-4515-85f0-dd95676c8382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:27.301465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5efc0e-baae-4515-85f0-dd95676c8382 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:27.310722 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1407/2828] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:27.586221 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc20d0-1d47-4a3e-b05a-17204c42c6fa None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:27.587798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-62cc20d0-1d47-4a3e-b05a-17204c42c6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:27.600064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc20d0-1d47-4a3e-b05a-17204c42c6fa None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:27.604698 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:27.605329 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:27.605640 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:27.605912 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] PATH_INFO: `/v3/projects/cd315fc7e8cf415e84a546002524ea4a` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:27.607017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd315fc7e8cf415e84a546002524ea4a)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:27.608930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:27.655043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd315fc7e8cf415e84a546002524ea4a', 'request_id': 'req-518eeca6-7315-470c-8564-f00d588b4026'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.661852 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd315fc7e8cf415e84a546002524ea4a', 'request_id': 'req-518eeca6-7315-470c-8564-f00d588b4026'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.667984 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd315fc7e8cf415e84a546002524ea4a', 'request_id': 'req-518eeca6-7315-470c-8564-f00d588b4026'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.678785 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] The token cache is being invalidate because project cd315fc7e8cf415e84a546002524ea4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:27.679072 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.679709 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.681208 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1422/2829] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:27 2022] DELETE /identity/v3/projects/cd315fc7e8cf415e84a546002524ea4a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:27.685293 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5efc0e-baae-4515-85f0-dd95676c8382 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:27.695904 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5efc0e-baae-4515-85f0-dd95676c8382 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:27.719790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:27.759920 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:27.760569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:27.760827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:27.761145 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] PATH_INFO: `/v3/users/cf992020c5f843edad12c8e97c238582` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:27.762357 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf992020c5f843edad12c8e97c238582)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:27.764404 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:27.813248 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf992020c5f843edad12c8e97c238582'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.813912 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf992020c5f843edad12c8e97c238582'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.822222 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf992020c5f843edad12c8e97c238582'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.829403 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf992020c5f843edad12c8e97c238582'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.829989 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf992020c5f843edad12c8e97c238582'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:27.846974 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1408/2830] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:27 2022] DELETE /identity/v3/users/cf992020c5f843edad12c8e97c238582 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:28.123269 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:28.124822 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:28.138226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518eeca6-7315-470c-8564-f00d588b4026 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:28.194587 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:28.195211 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:28.195486 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:28.195754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] PATH_INFO: `/v3/projects/a00c88d1d28b4c0bbf168e1311085b38` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:28.196846 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a00c88d1d28b4c0bbf168e1311085b38)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:28.198789 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:28.245228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a00c88d1d28b4c0bbf168e1311085b38', 'request_id': 'req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.246898 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:28.253750 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a00c88d1d28b4c0bbf168e1311085b38', 'request_id': 'req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.261199 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a00c88d1d28b4c0bbf168e1311085b38', 'request_id': 'req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.272000 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] The token cache is being invalidate because project a00c88d1d28b4c0bbf168e1311085b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:28.272304 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.272910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.274398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1423/2831] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:28 2022] DELETE /identity/v3/projects/a00c88d1d28b4c0bbf168e1311085b38 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:28.278363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:28.279788 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:28.292739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878d536-2c57-4641-9b91-58ea761a08a9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:28.348308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:28.348938 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:28.349216 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:28.349487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] PATH_INFO: `/v3/users/bc7054b22a364d36acd4fe4f76fa11c8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:28.350654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7054b22a364d36acd4fe4f76fa11c8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:28.352545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:28.395392 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7054b22a364d36acd4fe4f76fa11c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.396027 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7054b22a364d36acd4fe4f76fa11c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.397199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:28.405207 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7054b22a364d36acd4fe4f76fa11c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.413049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7054b22a364d36acd4fe4f76fa11c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.413975 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-fb534ca9-de87-4c88-b43e-26a55fd9d7a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7054b22a364d36acd4fe4f76fa11c8'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.431179 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1409/2832] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:28 2022] DELETE /identity/v3/users/bc7054b22a364d36acd4fe4f76fa11c8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:28.725111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:28.726543 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:28.739566 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f5b02-4e6d-48dd-aeae-dbe66f8ee228 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:28.803217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:28.803936 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:28.804214 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:28.804482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] PATH_INFO: `/v3/projects/d63a5a94c26d4e85aa50612d280347b4` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:28.805721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63a5a94c26d4e85aa50612d280347b4)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:28.807627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:28.861482 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63a5a94c26d4e85aa50612d280347b4', 'request_id': 'req-5caa08df-d04c-4c4a-a56b-3a12d8000416'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.878433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63a5a94c26d4e85aa50612d280347b4', 'request_id': 'req-5caa08df-d04c-4c4a-a56b-3a12d8000416'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.885394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63a5a94c26d4e85aa50612d280347b4', 'request_id': 'req-5caa08df-d04c-4c4a-a56b-3a12d8000416'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.896510 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] The token cache is being invalidate because project d63a5a94c26d4e85aa50612d280347b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:28.896728 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.897356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5caa08df-d04c-4c4a-a56b-3a12d8000416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:28.898900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1424/2833] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:28 2022] DELETE /identity/v3/projects/d63a5a94c26d4e85aa50612d280347b4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:28.905013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f28b5-7fbf-490d-8526-43ad2be05e59 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:28.905387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f28b5-7fbf-490d-8526-43ad2be05e59 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:28.905770 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f28b5-7fbf-490d-8526-43ad2be05e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:28.926341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-330f28b5-7fbf-490d-8526-43ad2be05e59 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:28.927524 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-330f28b5-7fbf-490d-8526-43ad2be05e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:28.994807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1410/2834] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:29.003463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7ac5b-e00c-4d84-86d0-6c72a3a58414 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:29.004249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7ac5b-e00c-4d84-86d0-6c72a3a58414 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:29.004522 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7ac5b-e00c-4d84-86d0-6c72a3a58414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:29.025747 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-47d7ac5b-e00c-4d84-86d0-6c72a3a58414 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:29.026940 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-47d7ac5b-e00c-4d84-86d0-6c72a3a58414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:29.035018 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1425/2835] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:29.039323 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f28b5-7fbf-490d-8526-43ad2be05e59 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:29.048957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f28b5-7fbf-490d-8526-43ad2be05e59 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:29.053571 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd599b-e54b-43d3-afec-f18d6296c26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:29.054191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd599b-e54b-43d3-afec-f18d6296c26a admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:29.054466 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd599b-e54b-43d3-afec-f18d6296c26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:29.054735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd599b-e54b-43d3-afec-f18d6296c26a admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:29.055378 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd599b-e54b-43d3-afec-f18d6296c26a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:30:29.055673 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd599b-e54b-43d3-afec-f18d6296c26a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:29.057253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd599b-e54b-43d3-afec-f18d6296c26a admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:29.069672 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1411/2836] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:30:29.074049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7ac5b-e00c-4d84-86d0-6c72a3a58414 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:29.083976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7ac5b-e00c-4d84-86d0-6c72a3a58414 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:29.088601 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251b8d5-3fc9-4d77-955f-02ba58a36022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:29.089198 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251b8d5-3fc9-4d77-955f-02ba58a36022 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:29.089471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251b8d5-3fc9-4d77-955f-02ba58a36022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:29.089824 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251b8d5-3fc9-4d77-955f-02ba58a36022 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:29.090457 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251b8d5-3fc9-4d77-955f-02ba58a36022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:29.092375 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251b8d5-3fc9-4d77-955f-02ba58a36022 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:29.126791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1426/2837] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:29 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:29.131534 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd599b-e54b-43d3-afec-f18d6296c26a admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:29.143569 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd599b-e54b-43d3-afec-f18d6296c26a admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:29.209924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982114d-acf4-448c-a293-ca2a1249df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:29.210537 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982114d-acf4-448c-a293-ca2a1249df01 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:29.210816 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982114d-acf4-448c-a293-ca2a1249df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:29.211202 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982114d-acf4-448c-a293-ca2a1249df01 admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:29.211835 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982114d-acf4-448c-a293-ca2a1249df01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:29.213972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982114d-acf4-448c-a293-ca2a1249df01 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:29.239315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1412/2838] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:29 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:29.243519 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251b8d5-3fc9-4d77-955f-02ba58a36022 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:29.255924 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251b8d5-3fc9-4d77-955f-02ba58a36022 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:29.262275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c318c7-0002-4959-bec6-cd76dfbb0f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:29.262915 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c318c7-0002-4959-bec6-cd76dfbb0f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:29.263191 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c318c7-0002-4959-bec6-cd76dfbb0f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:29.263464 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c318c7-0002-4959-bec6-cd76dfbb0f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:29.264112 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c318c7-0002-4959-bec6-cd76dfbb0f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:29.268062 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c318c7-0002-4959-bec6-cd76dfbb0f6f admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:29.281798 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1427/2839] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:29.283657 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982114d-acf4-448c-a293-ca2a1249df01 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:29.294995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982114d-acf4-448c-a293-ca2a1249df01 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:29.299615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90796-ea07-43d8-b730-4e95f13c4a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:29.300212 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90796-ea07-43d8-b730-4e95f13c4a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:29.301049 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90796-ea07-43d8-b730-4e95f13c4a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:29.301349 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90796-ea07-43d8-b730-4e95f13c4a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:29.302054 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f90796-ea07-43d8-b730-4e95f13c4a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:29.303666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f90796-ea07-43d8-b730-4e95f13c4a3f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:29.314402 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1413/2840] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:30:29.319312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c318c7-0002-4959-bec6-cd76dfbb0f6f admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:29.330135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c318c7-0002-4959-bec6-cd76dfbb0f6f admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:29.335067 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c118102-e1bb-4326-8f71-719a9b05e42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:29.335716 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118102-e1bb-4326-8f71-719a9b05e42c admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:29.335992 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118102-e1bb-4326-8f71-719a9b05e42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:29.336264 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118102-e1bb-4326-8f71-719a9b05e42c admin admin] PATH_INFO: `/v3/projects/ba9a9a9925dd46338c6f318da67bc103/users/2cb7d69fec1348f09a0cb7d6dfd889f7/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:29.351129 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c118102-e1bb-4326-8f71-719a9b05e42c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9a9a9925dd46338c6f318da67bc103, user_id=2cb7d69fec1348f09a0cb7d6dfd889f7, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:29.353810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c118102-e1bb-4326-8f71-719a9b05e42c admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:29.367190 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1428/2841] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:29 2022] PUT /identity/v3/projects/ba9a9a9925dd46338c6f318da67bc103/users/2cb7d69fec1348f09a0cb7d6dfd889f7/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:29.380237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b015f-1899-4ed2-9786-cfe36b0a0e54 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:29.380521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b015f-1899-4ed2-9786-cfe36b0a0e54 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:29.380790 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b015f-1899-4ed2-9786-cfe36b0a0e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:29.414609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8e7b015f-1899-4ed2-9786-cfe36b0a0e54 None None] MFA Rules not processed for user `2cb7d69fec1348f09a0cb7d6dfd889f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:29.415873 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8e7b015f-1899-4ed2-9786-cfe36b0a0e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:29.434995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:29.484405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1414/2842] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:29.496620 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c118102-e1bb-4326-8f71-719a9b05e42c admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:29.498403 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4c118102-e1bb-4326-8f71-719a9b05e42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:29.503268 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:29.571627 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c118102-e1bb-4326-8f71-719a9b05e42c admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:29.575405 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6891128-8af0-4d7e-9cc4-31273a606463 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:29.576008 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6891128-8af0-4d7e-9cc4-31273a606463 service cinder] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:29.576285 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6891128-8af0-4d7e-9cc4-31273a606463 service cinder] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:29.576553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6891128-8af0-4d7e-9cc4-31273a606463 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:29.584934 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6891128-8af0-4d7e-9cc4-31273a606463 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:29.586397 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6891128-8af0-4d7e-9cc4-31273a606463 service cinder] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:29.595559 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1429/2843] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:31.860972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b015f-1899-4ed2-9786-cfe36b0a0e54 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:31.862476 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8e7b015f-1899-4ed2-9786-cfe36b0a0e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:31.922518 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b015f-1899-4ed2-9786-cfe36b0a0e54 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:31.926333 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcd281-7e4b-4b10-9dbc-ee4a5a18d0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:31.926944 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcd281-7e4b-4b10-9dbc-ee4a5a18d0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:31.927223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcd281-7e4b-4b10-9dbc-ee4a5a18d0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:31.927496 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcd281-7e4b-4b10-9dbc-ee4a5a18d0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:31.929235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-73fcd281-7e4b-4b10-9dbc-ee4a5a18d0d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:31.994141 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcd281-7e4b-4b10-9dbc-ee4a5a18d0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:31.995665 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcd281-7e4b-4b10-9dbc-ee4a5a18d0d6 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:32.006148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1415/2844] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:32.236311 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6891128-8af0-4d7e-9cc4-31273a606463 service cinder] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:32.343641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6891128-8af0-4d7e-9cc4-31273a606463 service cinder] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:32.349737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:32.350448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:32.350758 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:32.351083 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] PATH_INFO: `/v3/users/2cb7d69fec1348f09a0cb7d6dfd889f7` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:32.352273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb7d69fec1348f09a0cb7d6dfd889f7)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:32.354360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:32.408957 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb7d69fec1348f09a0cb7d6dfd889f7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:32.409793 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb7d69fec1348f09a0cb7d6dfd889f7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:32.417319 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb7d69fec1348f09a0cb7d6dfd889f7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:32.418084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb7d69fec1348f09a0cb7d6dfd889f7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:32.427840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb7d69fec1348f09a0cb7d6dfd889f7'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:32.448871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1430/2845] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:32 2022] DELETE /identity/v3/users/2cb7d69fec1348f09a0cb7d6dfd889f7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:30:32.453306 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036694f1-5b18-4723-95e5-8ed739705df7 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:32.453615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036694f1-5b18-4723-95e5-8ed739705df7 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:32.453888 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036694f1-5b18-4723-95e5-8ed739705df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:32.476218 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-036694f1-5b18-4723-95e5-8ed739705df7 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:32.477363 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-036694f1-5b18-4723-95e5-8ed739705df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:32.560850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1416/2846] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:32.572923 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:32.574786 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:32.602695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:32.654754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660f256-ad21-4af2-8c28-20a12f5feee8 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:32.659288 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38e1e7-6116-455c-8d3d-8726d9c01bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:32.660743 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38e1e7-6116-455c-8d3d-8726d9c01bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:32.661019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38e1e7-6116-455c-8d3d-8726d9c01bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:32.661290 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38e1e7-6116-455c-8d3d-8726d9c01bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:32.671862 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38e1e7-6116-455c-8d3d-8726d9c01bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:32.674497 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38e1e7-6116-455c-8d3d-8726d9c01bd1 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:32.690967 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1431/2847] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:33.180247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036694f1-5b18-4723-95e5-8ed739705df7 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:33.182327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-036694f1-5b18-4723-95e5-8ed739705df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:33.196365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036694f1-5b18-4723-95e5-8ed739705df7 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:33.205487 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:33.206184 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.206462 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.207273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] PATH_INFO: `/v3/projects/ba9a9a9925dd46338c6f318da67bc103` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.208615 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9a9a9925dd46338c6f318da67bc103)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:33.210870 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:33.218859 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=100 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:33.270591 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9a9a9925dd46338c6f318da67bc103', 'request_id': 'req-34ec8c47-a52e-40a9-9063-1804bc080e35'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:33.279394 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9a9a9925dd46338c6f318da67bc103', 'request_id': 'req-34ec8c47-a52e-40a9-9063-1804bc080e35'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:33.287780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9a9a9925dd46338c6f318da67bc103', 'request_id': 'req-34ec8c47-a52e-40a9-9063-1804bc080e35'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:33.299740 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] The token cache is being invalidate because project ba9a9a9925dd46338c6f318da67bc103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:33.300314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:33.301014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-34ec8c47-a52e-40a9-9063-1804bc080e35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:33.302795 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1417/2848] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:33 2022] DELETE /identity/v3/projects/ba9a9a9925dd46338c6f318da67bc103 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:30:33.307569 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7494b-18cd-4609-bb0a-78988c58aea0 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.308038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7494b-18cd-4609-bb0a-78988c58aea0 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.308263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7494b-18cd-4609-bb0a-78988c58aea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.333194 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-bba7494b-18cd-4609-bb0a-78988c58aea0 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:33.334799 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-bba7494b-18cd-4609-bb0a-78988c58aea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:33.411230 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1432/2849] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:33.426084 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544df18-c1d1-4df9-a709-7a5d45763ebd None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.426587 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544df18-c1d1-4df9-a709-7a5d45763ebd None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.427089 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544df18-c1d1-4df9-a709-7a5d45763ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.460063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-4544df18-c1d1-4df9-a709-7a5d45763ebd None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:33.463489 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-4544df18-c1d1-4df9-a709-7a5d45763ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:33.476701 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1418/2850] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:33.481689 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7494b-18cd-4609-bb0a-78988c58aea0 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:33.497251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7494b-18cd-4609-bb0a-78988c58aea0 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:33.505717 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54ffea-d842-4fb4-b31c-2002c0498d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:33.506527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ffea-d842-4fb4-b31c-2002c0498d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.506913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ffea-d842-4fb4-b31c-2002c0498d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.507289 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ffea-d842-4fb4-b31c-2002c0498d5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.508480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54ffea-d842-4fb4-b31c-2002c0498d5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:30:33.508892 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54ffea-d842-4fb4-b31c-2002c0498d5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:33.511019 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54ffea-d842-4fb4-b31c-2002c0498d5b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:33.532968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1433/2851] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:30:33.536995 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544df18-c1d1-4df9-a709-7a5d45763ebd None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:33.548422 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544df18-c1d1-4df9-a709-7a5d45763ebd None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:33.554594 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06687ea-a44b-49ef-8174-0a2932e8f086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:33.555210 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06687ea-a44b-49ef-8174-0a2932e8f086 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.555488 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06687ea-a44b-49ef-8174-0a2932e8f086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.555758 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06687ea-a44b-49ef-8174-0a2932e8f086 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.556651 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06687ea-a44b-49ef-8174-0a2932e8f086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:33.558618 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06687ea-a44b-49ef-8174-0a2932e8f086 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:33.595277 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1419/2852] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:33 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:33.599177 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54ffea-d842-4fb4-b31c-2002c0498d5b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:33.611844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54ffea-d842-4fb4-b31c-2002c0498d5b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:33.678532 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c1e38-5628-447f-b985-5de231e388ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:33.678830 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c1e38-5628-447f-b985-5de231e388ea admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.679133 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c1e38-5628-447f-b985-5de231e388ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.679398 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c1e38-5628-447f-b985-5de231e388ea admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.680149 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c1e38-5628-447f-b985-5de231e388ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:33.682420 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c1e38-5628-447f-b985-5de231e388ea admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:33.709031 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1434/2853] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:33 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:33.712926 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06687ea-a44b-49ef-8174-0a2932e8f086 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:33.723751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06687ea-a44b-49ef-8174-0a2932e8f086 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:33.728907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1383c1-6696-469f-9171-6366c1404fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:33.731123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1383c1-6696-469f-9171-6366c1404fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.731386 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1383c1-6696-469f-9171-6366c1404fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.731729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1383c1-6696-469f-9171-6366c1404fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.732545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1383c1-6696-469f-9171-6366c1404fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:33.734165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1383c1-6696-469f-9171-6366c1404fd5 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:33.747255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1420/2854] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:33.752197 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c1e38-5628-447f-b985-5de231e388ea admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:33.766171 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c1e38-5628-447f-b985-5de231e388ea admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:33.775350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfc2b5-86ab-4361-8a71-774e74c172bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:33.776091 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfc2b5-86ab-4361-8a71-774e74c172bf admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.776373 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfc2b5-86ab-4361-8a71-774e74c172bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.776701 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfc2b5-86ab-4361-8a71-774e74c172bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.777357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfc2b5-86ab-4361-8a71-774e74c172bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:33.781721 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfc2b5-86ab-4361-8a71-774e74c172bf admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:33.793610 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1435/2855] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:33.798045 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1383c1-6696-469f-9171-6366c1404fd5 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:33.812801 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1383c1-6696-469f-9171-6366c1404fd5 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:33.818966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86874723-e129-4ec6-a30a-26e4d476712b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:33.819632 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86874723-e129-4ec6-a30a-26e4d476712b admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.820165 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86874723-e129-4ec6-a30a-26e4d476712b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.820291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86874723-e129-4ec6-a30a-26e4d476712b admin admin] PATH_INFO: `/v3/projects/d48650468b294dc39afeddab84939924/users/f2265fddf2704b5991c63c0267ae1730/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.842127 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86874723-e129-4ec6-a30a-26e4d476712b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d48650468b294dc39afeddab84939924, user_id=f2265fddf2704b5991c63c0267ae1730, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:33.846767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86874723-e129-4ec6-a30a-26e4d476712b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:33.877663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1421/2856] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:33 2022] PUT /identity/v3/projects/d48650468b294dc39afeddab84939924/users/f2265fddf2704b5991c63c0267ae1730/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:30:33.880556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9580c-dfa7-4ae5-a0ac-9c2093ac3590 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:33.880556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9580c-dfa7-4ae5-a0ac-9c2093ac3590 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:33.880556 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9580c-dfa7-4ae5-a0ac-9c2093ac3590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:33.911877 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-d2d9580c-dfa7-4ae5-a0ac-9c2093ac3590 None None] MFA Rules not processed for user `f2265fddf2704b5991c63c0267ae1730`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:33.915921 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-d2d9580c-dfa7-4ae5-a0ac-9c2093ac3590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:33.930187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:34.001157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1436/2857] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:34.009557 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86874723-e129-4ec6-a30a-26e4d476712b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:34.014191 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-86874723-e129-4ec6-a30a-26e4d476712b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:34.094213 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86874723-e129-4ec6-a30a-26e4d476712b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:34.098305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42f008-f245-4703-a1a7-10fabb8af7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a6e8bc66986d48aa9dcbc19258ff6e1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:34.098924 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42f008-f245-4703-a1a7-10fabb8af7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:34.099681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42f008-f245-4703-a1a7-10fabb8af7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:34.100114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42f008-f245-4703-a1a7-10fabb8af7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:34.111110 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42f008-f245-4703-a1a7-10fabb8af7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:34.113385 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42f008-f245-4703-a1a7-10fabb8af7e9 service cinder] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:34.124371 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1422/2858] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:35.607989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7b5c-12cc-4ccb-9f84-b8b567c1b491 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:35.608513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7b5c-12cc-4ccb-9f84-b8b567c1b491 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:35.608513 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7b5c-12cc-4ccb-9f84-b8b567c1b491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:35.629176 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-09eb7b5c-12cc-4ccb-9f84-b8b567c1b491 None None] MFA Rules not processed for user `f2265fddf2704b5991c63c0267ae1730`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:35.630490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-09eb7b5c-12cc-4ccb-9f84-b8b567c1b491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:35.638049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1437/2859] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:35.758540 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42f008-f245-4703-a1a7-10fabb8af7e9 service cinder] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:35.760059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7a42f008-f245-4703-a1a7-10fabb8af7e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:35.804205 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:35.841575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42f008-f245-4703-a1a7-10fabb8af7e9 service cinder] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:35.846490 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:35.847686 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:35.847969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:35.848238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] PATH_INFO: `/v3/users/bef3afe7e66447ac9f603a68d5b5cac7` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:35.849426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef3afe7e66447ac9f603a68d5b5cac7)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:35.851458 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:35.908972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef3afe7e66447ac9f603a68d5b5cac7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:35.910335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef3afe7e66447ac9f603a68d5b5cac7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:35.918545 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef3afe7e66447ac9f603a68d5b5cac7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:35.934155 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef3afe7e66447ac9f603a68d5b5cac7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:35.935119 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef3afe7e66447ac9f603a68d5b5cac7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:35.955314 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1423/2860] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:35 2022] DELETE /identity/v3/users/bef3afe7e66447ac9f603a68d5b5cac7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:35.960059 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76e023-7e88-491f-89c5-2ad1b35bb671 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:35.960343 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76e023-7e88-491f-89c5-2ad1b35bb671 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:35.960613 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76e023-7e88-491f-89c5-2ad1b35bb671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:35.985036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-aa76e023-7e88-491f-89c5-2ad1b35bb671 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:35.986275 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-aa76e023-7e88-491f-89c5-2ad1b35bb671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:36.063347 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1438/2861] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:36.073313 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:36.074899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:36.154621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fee78-f8cb-4d76-97ee-27527c200eb9 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:36.159137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d440642-19f0-478b-8d61-b121e764a114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:36.160398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d440642-19f0-478b-8d61-b121e764a114 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:36.160739 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d440642-19f0-478b-8d61-b121e764a114 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:36.161092 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d440642-19f0-478b-8d61-b121e764a114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:36.171972 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d440642-19f0-478b-8d61-b121e764a114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:36.173723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d440642-19f0-478b-8d61-b121e764a114 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:36.184792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1424/2862] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:36.527479 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76e023-7e88-491f-89c5-2ad1b35bb671 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:36.528945 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-aa76e023-7e88-491f-89c5-2ad1b35bb671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:36.547854 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76e023-7e88-491f-89c5-2ad1b35bb671 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:36.556249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:36.556897 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:36.557173 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:36.557447 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] PATH_INFO: `/v3/projects/cb23c05314cb497688b15e86f4a99d9a` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:36.559036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb23c05314cb497688b15e86f4a99d9a)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:36.561298 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:36.634036 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb23c05314cb497688b15e86f4a99d9a', 'request_id': 'req-2c78c4e4-bf14-4561-b976-70c2eb554319'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:36.642869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb23c05314cb497688b15e86f4a99d9a', 'request_id': 'req-2c78c4e4-bf14-4561-b976-70c2eb554319'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:36.650452 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb23c05314cb497688b15e86f4a99d9a', 'request_id': 'req-2c78c4e4-bf14-4561-b976-70c2eb554319'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:36.662553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] The token cache is being invalidate because project cb23c05314cb497688b15e86f4a99d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:36.662918 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:36.663857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2c78c4e4-bf14-4561-b976-70c2eb554319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:36.666182 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1439/2863] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:36 2022] DELETE /identity/v3/projects/cb23c05314cb497688b15e86f4a99d9a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:36.671290 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7867b-4df2-4c65-be70-938e8f01e5f3 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:36.671700 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7867b-4df2-4c65-be70-938e8f01e5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:36.672203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7867b-4df2-4c65-be70-938e8f01e5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:36.697735 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-9de7867b-4df2-4c65-be70-938e8f01e5f3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:36.699161 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-9de7867b-4df2-4c65-be70-938e8f01e5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:36.768521 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:36.785608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1425/2864] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:36.792273 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40300a0-ea00-4d3e-8825-6f276e09e017 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:36.792604 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40300a0-ea00-4d3e-8825-6f276e09e017 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:36.792863 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40300a0-ea00-4d3e-8825-6f276e09e017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:36.818088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-e40300a0-ea00-4d3e-8825-6f276e09e017 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:36.819487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-e40300a0-ea00-4d3e-8825-6f276e09e017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:36.830487 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1440/2865] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:36.836666 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7867b-4df2-4c65-be70-938e8f01e5f3 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:36.846850 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7867b-4df2-4c65-be70-938e8f01e5f3 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:36.852013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0fcaa-6dba-4bf2-9a24-9aed46126672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:36.852861 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0fcaa-6dba-4bf2-9a24-9aed46126672 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:36.853169 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0fcaa-6dba-4bf2-9a24-9aed46126672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:36.853575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0fcaa-6dba-4bf2-9a24-9aed46126672 admin admin] PATH_INFO: `/v3/domains` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:36.854497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0fcaa-6dba-4bf2-9a24-9aed46126672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81337) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:30:36.854902 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0fcaa-6dba-4bf2-9a24-9aed46126672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:36.857472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0fcaa-6dba-4bf2-9a24-9aed46126672 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:36.879189 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1426/2866] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:30:36.883718 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40300a0-ea00-4d3e-8825-6f276e09e017 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:36.895644 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40300a0-ea00-4d3e-8825-6f276e09e017 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:36.902437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33d13f-3bdc-480e-a6d1-f119895d6fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:36.903049 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33d13f-3bdc-480e-a6d1-f119895d6fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:36.903480 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33d13f-3bdc-480e-a6d1-f119895d6fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:36.903752 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33d13f-3bdc-480e-a6d1-f119895d6fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:36.904458 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33d13f-3bdc-480e-a6d1-f119895d6fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:36.907146 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33d13f-3bdc-480e-a6d1-f119895d6fb9 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:36.938625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1441/2867] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:36 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:36.944168 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0fcaa-6dba-4bf2-9a24-9aed46126672 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:36.954520 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0fcaa-6dba-4bf2-9a24-9aed46126672 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:37.016365 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48f4bd-794b-4cbf-b8e0-3a0216ab2cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:37.017003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48f4bd-794b-4cbf-b8e0-3a0216ab2cfe admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:37.017327 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48f4bd-794b-4cbf-b8e0-3a0216ab2cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:37.017641 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48f4bd-794b-4cbf-b8e0-3a0216ab2cfe admin admin] PATH_INFO: `/v3/users` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:37.018253 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48f4bd-794b-4cbf-b8e0-3a0216ab2cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:37.020240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48f4bd-794b-4cbf-b8e0-3a0216ab2cfe admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:37.046510 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1427/2868] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:36 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:37.050733 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33d13f-3bdc-480e-a6d1-f119895d6fb9 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:37.057677 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:37.065017 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33d13f-3bdc-480e-a6d1-f119895d6fb9 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:37.073094 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc78c3-0274-4e0a-8de6-0ea63dfcba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:37.073869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc78c3-0274-4e0a-8de6-0ea63dfcba70 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:37.074207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc78c3-0274-4e0a-8de6-0ea63dfcba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:37.074597 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc78c3-0274-4e0a-8de6-0ea63dfcba70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:37.075536 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc78c3-0274-4e0a-8de6-0ea63dfcba70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:37.077594 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc78c3-0274-4e0a-8de6-0ea63dfcba70 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:37.093550 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1442/2869] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 20:30:37.095793 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48f4bd-794b-4cbf-b8e0-3a0216ab2cfe admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:37.107412 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48f4bd-794b-4cbf-b8e0-3a0216ab2cfe admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:37.113595 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11e9f0-ed9c-429b-9a8f-9c5fd36e1d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:37.114201 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11e9f0-ed9c-429b-9a8f-9c5fd36e1d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:37.114478 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11e9f0-ed9c-429b-9a8f-9c5fd36e1d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:37.114747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11e9f0-ed9c-429b-9a8f-9c5fd36e1d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:37.115387 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11e9f0-ed9c-429b-9a8f-9c5fd36e1d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:37.116449 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc78c3-0274-4e0a-8de6-0ea63dfcba70 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:37.118463 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b7fc78c3-0274-4e0a-8de6-0ea63dfcba70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:37.118777 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11e9f0-ed9c-429b-9a8f-9c5fd36e1d2b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:37.131116 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1428/2870] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:37.134783 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11e9f0-ed9c-429b-9a8f-9c5fd36e1d2b admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:37.147063 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11e9f0-ed9c-429b-9a8f-9c5fd36e1d2b admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:37.154391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf3bea-813c-4802-8d5b-30ec289db92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:37.155543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf3bea-813c-4802-8d5b-30ec289db92f admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:37.155986 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf3bea-813c-4802-8d5b-30ec289db92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:37.156418 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf3bea-813c-4802-8d5b-30ec289db92f admin admin] PATH_INFO: `/v3/projects/512f3d98bc5e4481becbb60fcf9f1ea1/users/b5bba7d3b8824124ad900a8a20e499f7/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:37.176318 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf3bea-813c-4802-8d5b-30ec289db92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=512f3d98bc5e4481becbb60fcf9f1ea1, user_id=b5bba7d3b8824124ad900a8a20e499f7, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:37.179059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf3bea-813c-4802-8d5b-30ec289db92f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:37.195178 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1429/2871] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:37 2022] PUT /identity/v3/projects/512f3d98bc5e4481becbb60fcf9f1ea1/users/b5bba7d3b8824124ad900a8a20e499f7/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:30:37.205235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91fb7e-1c72-4c68-9357-82f97fd8348e None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:37.205544 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91fb7e-1c72-4c68-9357-82f97fd8348e None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:37.205820 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91fb7e-1c72-4c68-9357-82f97fd8348e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:37.224641 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc78c3-0274-4e0a-8de6-0ea63dfcba70 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:37.238982 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-6e91fb7e-1c72-4c68-9357-82f97fd8348e None None] MFA Rules not processed for user `b5bba7d3b8824124ad900a8a20e499f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:37.240406 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6e91fb7e-1c72-4c68-9357-82f97fd8348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:37.291768 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224dd15-42b9-44a7-ba1d-1d6dff871671 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:37.292374 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224dd15-42b9-44a7-ba1d-1d6dff871671 service placement] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:37.292649 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224dd15-42b9-44a7-ba1d-1d6dff871671 service placement] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:37.292919 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224dd15-42b9-44a7-ba1d-1d6dff871671 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:37.294826 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-b224dd15-42b9-44a7-ba1d-1d6dff871671 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:37.308593 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224dd15-42b9-44a7-ba1d-1d6dff871671 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:37.310257 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224dd15-42b9-44a7-ba1d-1d6dff871671 service placement] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:37.319057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1430/2872] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 20:30:37.322340 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1443/2873] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:37.327154 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91fb7e-1c72-4c68-9357-82f97fd8348e None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:37.328579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6e91fb7e-1c72-4c68-9357-82f97fd8348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:37.361433 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:37.392307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91fb7e-1c72-4c68-9357-82f97fd8348e None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:37.396619 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68300fdc-96e1-44b2-abf6-94a97bd29177 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47bafea2367e4c7992354bb2c23e7d57', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:37.397305 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300fdc-96e1-44b2-abf6-94a97bd29177 service glance] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:37.397681 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300fdc-96e1-44b2-abf6-94a97bd29177 service glance] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:37.398013 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68300fdc-96e1-44b2-abf6-94a97bd29177 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:37.407438 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68300fdc-96e1-44b2-abf6-94a97bd29177 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:37.408929 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68300fdc-96e1-44b2-abf6-94a97bd29177 service glance] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:37.418235 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1431/2874] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:47.316879 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=71 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:47.413874 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=14 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:49.663821 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7543-c225-49ef-9906-f59f525d2bf7 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:49.664282 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7543-c225-49ef-9906-f59f525d2bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:49.664386 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf7543-c225-49ef-9906-f59f525d2bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:49.694979 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-afdf7543-c225-49ef-9906-f59f525d2bf7 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:49.696157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-afdf7543-c225-49ef-9906-f59f525d2bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:49.748157 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1444/2875] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:30:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:49.793259 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68300fdc-96e1-44b2-abf6-94a97bd29177 service glance] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:49.794674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-68300fdc-96e1-44b2-abf6-94a97bd29177 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:49.866772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68300fdc-96e1-44b2-abf6-94a97bd29177 service glance] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:49.871766 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17652c74-f9e2-4c6d-b1e1-589221abaa7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:49.872410 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17652c74-f9e2-4c6d-b1e1-589221abaa7f service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:49.872685 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17652c74-f9e2-4c6d-b1e1-589221abaa7f service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:49.872955 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17652c74-f9e2-4c6d-b1e1-589221abaa7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:49.882002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17652c74-f9e2-4c6d-b1e1-589221abaa7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:49.883572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17652c74-f9e2-4c6d-b1e1-589221abaa7f service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:49.892456 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1432/2876] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:30:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:30:50.376261 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadd28c-8a40-4821-ac10-9eb780041282 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:50.376540 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadd28c-8a40-4821-ac10-9eb780041282 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:50.376816 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadd28c-8a40-4821-ac10-9eb780041282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:50.397147 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-dbadd28c-8a40-4821-ac10-9eb780041282 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:50.398365 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-dbadd28c-8a40-4821-ac10-9eb780041282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:50.405322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1445/2877] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 20:30:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:50.417947 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17652c74-f9e2-4c6d-b1e1-589221abaa7f service swift] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:50.428423 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17652c74-f9e2-4c6d-b1e1-589221abaa7f service swift] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:50.433391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b89078-1f23-4097-b45a-196def62bbc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:50.434006 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b89078-1f23-4097-b45a-196def62bbc5 service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:50.434280 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b89078-1f23-4097-b45a-196def62bbc5 service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:50.434549 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b89078-1f23-4097-b45a-196def62bbc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:50.443079 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b89078-1f23-4097-b45a-196def62bbc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:50.444684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b89078-1f23-4097-b45a-196def62bbc5 service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:50.452553 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1433/2878] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:30:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 05 20:30:50.669989 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ac56-7ab3-4eef-b0f3-e2ec6d211c90 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:50.670477 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ac56-7ab3-4eef-b0f3-e2ec6d211c90 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:50.670602 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ac56-7ab3-4eef-b0f3-e2ec6d211c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:50.690029 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-6a79ac56-7ab3-4eef-b0f3-e2ec6d211c90 None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:50.691226 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-6a79ac56-7ab3-4eef-b0f3-e2ec6d211c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:50.699251 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1446/2879] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 20:30:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:50.711391 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b89078-1f23-4097-b45a-196def62bbc5 service swift] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:50.721934 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b89078-1f23-4097-b45a-196def62bbc5 service swift] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:50.726827 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd1324-91dd-4eec-9070-f5570b3b3f34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:50.727409 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd1324-91dd-4eec-9070-f5570b3b3f34 service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:50.727684 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd1324-91dd-4eec-9070-f5570b3b3f34 service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:50.727952 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd1324-91dd-4eec-9070-f5570b3b3f34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:50.735674 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd1324-91dd-4eec-9070-f5570b3b3f34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:50.737227 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd1324-91dd-4eec-9070-f5570b3b3f34 service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:50.745074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1434/2880] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:30:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 20:30:51.058228 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece9306-7ed7-486e-af07-0f6cc27a95af None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:51.058509 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece9306-7ed7-486e-af07-0f6cc27a95af None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:51.058776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece9306-7ed7-486e-af07-0f6cc27a95af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:51.078322 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-5ece9306-7ed7-486e-af07-0f6cc27a95af None None] MFA Rules not processed for user `2314d9416eea4856aec5ac2a0cdfdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:51.079516 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5ece9306-7ed7-486e-af07-0f6cc27a95af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:51.086724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1447/2881] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 20:30:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:51.098648 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd1324-91dd-4eec-9070-f5570b3b3f34 service swift] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:51.109008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd1324-91dd-4eec-9070-f5570b3b3f34 service swift] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:51.113961 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badda1dc-a05d-4e5e-b3a3-f3ef5d544436 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8dec1dbdcd94316aad25c93c313a5a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:51.115220 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badda1dc-a05d-4e5e-b3a3-f3ef5d544436 service swift] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:51.115497 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badda1dc-a05d-4e5e-b3a3-f3ef5d544436 service swift] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:51.115774 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badda1dc-a05d-4e5e-b3a3-f3ef5d544436 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:51.123506 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badda1dc-a05d-4e5e-b3a3-f3ef5d544436 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:51.125042 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badda1dc-a05d-4e5e-b3a3-f3ef5d544436 service swift] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:51.133465 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1435/2882] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 20:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 05 20:30:53.229300 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece9306-7ed7-486e-af07-0f6cc27a95af None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:53.232757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-5ece9306-7ed7-486e-af07-0f6cc27a95af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:53.306314 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece9306-7ed7-486e-af07-0f6cc27a95af None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:53.311154 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:53.311810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:53.312089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:53.312360 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] PATH_INFO: `/v3/users/f2265fddf2704b5991c63c0267ae1730` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:53.313527 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2265fddf2704b5991c63c0267ae1730)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:53.315490 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:53.371541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2265fddf2704b5991c63c0267ae1730'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.371541 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2265fddf2704b5991c63c0267ae1730'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.371926 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2265fddf2704b5991c63c0267ae1730'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.372567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2265fddf2704b5991c63c0267ae1730'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.379899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2265fddf2704b5991c63c0267ae1730'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.391081 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:53.399193 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1448/2883] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:30:53 2022] DELETE /identity/v3/users/f2265fddf2704b5991c63c0267ae1730 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:53.403631 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aef680-cccc-4be9-9c46-304219b5749a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:53.404114 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aef680-cccc-4be9-9c46-304219b5749a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:53.404221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aef680-cccc-4be9-9c46-304219b5749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:53.423173 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-00aef680-cccc-4be9-9c46-304219b5749a None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:53.424341 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-00aef680-cccc-4be9-9c46-304219b5749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:53.481291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1436/2884] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:53.490754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:53.492517 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:53.552339 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7addd-600a-4290-ad79-049c92fdd484 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:53.556139 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc09288f-6b04-4f42-9945-c78ce2f21256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:53.556737 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09288f-6b04-4f42-9945-c78ce2f21256 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:53.557010 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09288f-6b04-4f42-9945-c78ce2f21256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:53.557274 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc09288f-6b04-4f42-9945-c78ce2f21256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:53.567069 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09288f-6b04-4f42-9945-c78ce2f21256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:53.569023 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc09288f-6b04-4f42-9945-c78ce2f21256 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:53.578906 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1449/2885] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:30:53.748272 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aef680-cccc-4be9-9c46-304219b5749a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:53.749723 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-00aef680-cccc-4be9-9c46-304219b5749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:53.763158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aef680-cccc-4be9-9c46-304219b5749a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:53.768138 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:53.768769 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:53.769574 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:53.769854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] PATH_INFO: `/v3/projects/d48650468b294dc39afeddab84939924` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:53.771807 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d48650468b294dc39afeddab84939924)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:53.773879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:53.818331 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:53.821353 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd48650468b294dc39afeddab84939924', 'request_id': 'req-cf266a09-d475-4d55-886f-9ff78500faf2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.828654 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd48650468b294dc39afeddab84939924', 'request_id': 'req-cf266a09-d475-4d55-886f-9ff78500faf2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.835563 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd48650468b294dc39afeddab84939924', 'request_id': 'req-cf266a09-d475-4d55-886f-9ff78500faf2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.846159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] The token cache is being invalidate because project d48650468b294dc39afeddab84939924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81337) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:30:53.846444 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.847065 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-cf266a09-d475-4d55-886f-9ff78500faf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:30:53.848512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1437/2886] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:30:53 2022] DELETE /identity/v3/projects/d48650468b294dc39afeddab84939924 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:53.853695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3049-4d50-49a1-85a7-9b5046ff38bd None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:53.853986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3049-4d50-49a1-85a7-9b5046ff38bd None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:53.854263 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3049-4d50-49a1-85a7-9b5046ff38bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:53.873653 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-c25e3049-4d50-49a1-85a7-9b5046ff38bd None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:53.874864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c25e3049-4d50-49a1-85a7-9b5046ff38bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:53.933336 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1450/2887] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:53.942969 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd40a65-4a5d-4913-86f3-b38ebab78c53 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:53.943247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd40a65-4a5d-4913-86f3-b38ebab78c53 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:53.943517 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd40a65-4a5d-4913-86f3-b38ebab78c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:53.963279 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-8fd40a65-4a5d-4913-86f3-b38ebab78c53 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:53.964468 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-8fd40a65-4a5d-4913-86f3-b38ebab78c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:53.972274 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1438/2888] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:53.976270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3049-4d50-49a1-85a7-9b5046ff38bd None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:53.985871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3049-4d50-49a1-85a7-9b5046ff38bd None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:53.990935 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a64c94-9d9c-40b2-8799-249523d8df61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:53.991466 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a64c94-9d9c-40b2-8799-249523d8df61 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:53.991741 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a64c94-9d9c-40b2-8799-249523d8df61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:53.992009 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a64c94-9d9c-40b2-8799-249523d8df61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:53.992652 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a64c94-9d9c-40b2-8799-249523d8df61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81338) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 20:30:53.992954 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a64c94-9d9c-40b2-8799-249523d8df61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:53.994582 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a64c94-9d9c-40b2-8799-249523d8df61 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:54.006715 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1451/2889] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 20:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 20:30:54.010637 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd40a65-4a5d-4913-86f3-b38ebab78c53 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:54.019732 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd40a65-4a5d-4913-86f3-b38ebab78c53 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:54.024308 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31529a38-7f00-44e7-a69f-6aeb05c024e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:54.024889 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31529a38-7f00-44e7-a69f-6aeb05c024e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:54.025162 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31529a38-7f00-44e7-a69f-6aeb05c024e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:54.025431 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31529a38-7f00-44e7-a69f-6aeb05c024e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:54.026090 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31529a38-7f00-44e7-a69f-6aeb05c024e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:54.028077 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31529a38-7f00-44e7-a69f-6aeb05c024e2 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:54.055522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1439/2890] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:54 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:54.059693 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a64c94-9d9c-40b2-8799-249523d8df61 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:54.070321 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a64c94-9d9c-40b2-8799-249523d8df61 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:54.124853 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44002aa5-3038-40d9-aa1b-8779eab71f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:54.125445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002aa5-3038-40d9-aa1b-8779eab71f10 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:54.125754 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002aa5-3038-40d9-aa1b-8779eab71f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:54.126024 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002aa5-3038-40d9-aa1b-8779eab71f10 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:54.126648 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44002aa5-3038-40d9-aa1b-8779eab71f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:54.128557 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44002aa5-3038-40d9-aa1b-8779eab71f10 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:54.152756 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1452/2891] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:54 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:54.153980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31529a38-7f00-44e7-a69f-6aeb05c024e2 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:54.164135 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31529a38-7f00-44e7-a69f-6aeb05c024e2 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:54.168669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a0534-13c9-4bc6-b925-d598a519006f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:54.169282 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a0534-13c9-4bc6-b925-d598a519006f admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:54.169582 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a0534-13c9-4bc6-b925-d598a519006f admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:54.169854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a0534-13c9-4bc6-b925-d598a519006f admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:54.170486 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a0534-13c9-4bc6-b925-d598a519006f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:54.172048 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a0534-13c9-4bc6-b925-d598a519006f admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:54.182244 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1440/2892] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:54.185956 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44002aa5-3038-40d9-aa1b-8779eab71f10 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:54.197598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44002aa5-3038-40d9-aa1b-8779eab71f10 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:54.202128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907fb7f-77ba-464a-bc7c-180dbe07c518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:54.202711 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907fb7f-77ba-464a-bc7c-180dbe07c518 admin admin] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:54.202985 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907fb7f-77ba-464a-bc7c-180dbe07c518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:54.203254 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907fb7f-77ba-464a-bc7c-180dbe07c518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:54.203880 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907fb7f-77ba-464a-bc7c-180dbe07c518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:54.205448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907fb7f-77ba-464a-bc7c-180dbe07c518 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:54.215757 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1453/2893] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:54.219895 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a0534-13c9-4bc6-b925-d598a519006f admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:54.229742 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a0534-13c9-4bc6-b925-d598a519006f admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:54.234498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18c3f8-f75c-4331-9c87-cf02c0d58eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:54.235170 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18c3f8-f75c-4331-9c87-cf02c0d58eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:54.235474 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18c3f8-f75c-4331-9c87-cf02c0d58eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:54.235747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18c3f8-f75c-4331-9c87-cf02c0d58eb7 admin admin] PATH_INFO: `/v3/projects/3dae2633ff47484981df67d10b94de47/users/7a5a70e8cad0452188dd1b750c4efa17/roles/84d6c85b41034cae925afeb8921381e8` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:54.250725 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18c3f8-f75c-4331-9c87-cf02c0d58eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dae2633ff47484981df67d10b94de47, user_id=7a5a70e8cad0452188dd1b750c4efa17, role_id=84d6c85b41034cae925afeb8921381e8)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:54.253285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18c3f8-f75c-4331-9c87-cf02c0d58eb7 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:54.266411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1441/2894] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:54 2022] PUT /identity/v3/projects/3dae2633ff47484981df67d10b94de47/users/7a5a70e8cad0452188dd1b750c4efa17/roles/84d6c85b41034cae925afeb8921381e8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:54.271380 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073b4df-8a26-44b9-8f6d-e245a84ba2d3 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:54.271844 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073b4df-8a26-44b9-8f6d-e245a84ba2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:54.271968 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073b4df-8a26-44b9-8f6d-e245a84ba2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:54.279869 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:54.292611 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-9073b4df-8a26-44b9-8f6d-e245a84ba2d3 None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:54.293858 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-9073b4df-8a26-44b9-8f6d-e245a84ba2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:54.350016 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1454/2895] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:54.359755 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18c3f8-f75c-4331-9c87-cf02c0d58eb7 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:54.361411 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ef18c3f8-f75c-4331-9c87-cf02c0d58eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:54.419010 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18c3f8-f75c-4331-9c87-cf02c0d58eb7 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:54.422627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b01f3-8dcc-4196-ac4e-32526394ee1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:54.423234 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b01f3-8dcc-4196-ac4e-32526394ee1f service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:54.423512 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b01f3-8dcc-4196-ac4e-32526394ee1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:54.423785 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b01f3-8dcc-4196-ac4e-32526394ee1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:54.433583 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b01f3-8dcc-4196-ac4e-32526394ee1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:54.434860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b01f3-8dcc-4196-ac4e-32526394ee1f service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:54.444759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1442/2896] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 20:30:58.713788 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2e5c4-1871-4c18-b297-125bbd10e545 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:58.714659 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2e5c4-1871-4c18-b297-125bbd10e545 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:58.714930 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2e5c4-1871-4c18-b297-125bbd10e545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:58.750441 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-30b2e5c4-1871-4c18-b297-125bbd10e545 None None] MFA Rules not processed for user `7a5a70e8cad0452188dd1b750c4efa17`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:30:58.751663 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-30b2e5c4-1871-4c18-b297-125bbd10e545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:30:58.821745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1455/2897] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:30:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:30:58.826310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b01f3-8dcc-4196-ac4e-32526394ee1f service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:58.836744 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b01f3-8dcc-4196-ac4e-32526394ee1f service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:58.841323 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbc286-1404-451e-a9c2-4a6bb6e58246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:58.841958 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc286-1404-451e-a9c2-4a6bb6e58246 admin admin] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:58.842231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc286-1404-451e-a9c2-4a6bb6e58246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:58.842498 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbc286-1404-451e-a9c2-4a6bb6e58246 admin admin] PATH_INFO: `/v3/projects` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:58.843132 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbc286-1404-451e-a9c2-4a6bb6e58246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:58.845061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbc286-1404-451e-a9c2-4a6bb6e58246 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:58.893047 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1443/2898] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 20:30:58 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 20:30:58.897833 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2e5c4-1871-4c18-b297-125bbd10e545 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:58.910802 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2e5c4-1871-4c18-b297-125bbd10e545 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:58.990350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e0ea9-9a95-44b3-8586-fc569de6ca19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:58.991252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e0ea9-9a95-44b3-8586-fc569de6ca19 admin admin] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:58.991695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e0ea9-9a95-44b3-8586-fc569de6ca19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:58.992114 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e0ea9-9a95-44b3-8586-fc569de6ca19 admin admin] PATH_INFO: `/v3/users` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:58.993407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e0ea9-9a95-44b3-8586-fc569de6ca19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:58.996001 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e0ea9-9a95-44b3-8586-fc569de6ca19 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:59.030871 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1456/2899] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 20:30:58 2022] POST /identity/v3/users => generated 295 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 20:30:59.034752 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbc286-1404-451e-a9c2-4a6bb6e58246 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:59.047899 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbc286-1404-451e-a9c2-4a6bb6e58246 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:59.053356 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60264-5126-4732-816e-23375c875628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:59.054500 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60264-5126-4732-816e-23375c875628 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:59.054903 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60264-5126-4732-816e-23375c875628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:59.055401 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60264-5126-4732-816e-23375c875628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:59.056209 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad60264-5126-4732-816e-23375c875628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:59.058439 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad60264-5126-4732-816e-23375c875628 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:59.070288 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1444/2900] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:59.074963 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e0ea9-9a95-44b3-8586-fc569de6ca19 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:59.087776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e0ea9-9a95-44b3-8586-fc569de6ca19 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:59.093774 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca623bb-e323-4bf6-adc5-b05fb07754ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:59.094567 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca623bb-e323-4bf6-adc5-b05fb07754ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:59.094705 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca623bb-e323-4bf6-adc5-b05fb07754ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:59.094976 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca623bb-e323-4bf6-adc5-b05fb07754ec admin admin] PATH_INFO: `/v3/projects/9b9a9e5e833c4d19bd06606517350d22/users/7fd3a2891f734a659d52f308270bf6ce/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:59.112532 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca623bb-e323-4bf6-adc5-b05fb07754ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9a9e5e833c4d19bd06606517350d22, user_id=7fd3a2891f734a659d52f308270bf6ce, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:59.115434 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca623bb-e323-4bf6-adc5-b05fb07754ec admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:59.131135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:59.136500 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1457/2901] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:59 2022] PUT /identity/v3/projects/9b9a9e5e833c4d19bd06606517350d22/users/7fd3a2891f734a659d52f308270bf6ce/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:30:59.140522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60264-5126-4732-816e-23375c875628 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:59.161405 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60264-5126-4732-816e-23375c875628 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:59.210059 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:30:59.231504 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302d75-8a5a-436d-94c6-a99aecdc4983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:59.232031 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302d75-8a5a-436d-94c6-a99aecdc4983 admin admin] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:59.232307 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302d75-8a5a-436d-94c6-a99aecdc4983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:59.232579 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302d75-8a5a-436d-94c6-a99aecdc4983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:59.233217 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec302d75-8a5a-436d-94c6-a99aecdc4983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:59.235008 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec302d75-8a5a-436d-94c6-a99aecdc4983 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:59.246231 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1445/2902] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 20:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 20:30:59.250363 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca623bb-e323-4bf6-adc5-b05fb07754ec admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:30:59.261749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca623bb-e323-4bf6-adc5-b05fb07754ec admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:30:59.266531 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589b514-6b5e-4ff2-a1da-13f2b2eaa18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:30:59.267260 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589b514-6b5e-4ff2-a1da-13f2b2eaa18e admin admin] REQUEST_METHOD: `PUT` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:30:59.267749 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589b514-6b5e-4ff2-a1da-13f2b2eaa18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:30:59.267885 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589b514-6b5e-4ff2-a1da-13f2b2eaa18e admin admin] PATH_INFO: `/v3/projects/9b9a9e5e833c4d19bd06606517350d22/users/7fd3a2891f734a659d52f308270bf6ce/roles/36331ec562ff42f883fc883363bc4c8d` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:30:59.269990 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589b514-6b5e-4ff2-a1da-13f2b2eaa18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9a9e5e833c4d19bd06606517350d22, user_id=7fd3a2891f734a659d52f308270bf6ce, role_id=36331ec562ff42f883fc883363bc4c8d)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:30:59.272738 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589b514-6b5e-4ff2-a1da-13f2b2eaa18e admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:30:59.283675 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1458/2903] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 20:30:59 2022] PUT /identity/v3/projects/9b9a9e5e833c4d19bd06606517350d22/users/7fd3a2891f734a659d52f308270bf6ce/roles/36331ec562ff42f883fc883363bc4c8d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:31:02.512262 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302d75-8a5a-436d-94c6-a99aecdc4983 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:31:02.514784 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ec302d75-8a5a-436d-94c6-a99aecdc4983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:02.531414 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302d75-8a5a-436d-94c6-a99aecdc4983 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:31:02.654721 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:31:02.655677 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:31:02.656123 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:31:02.656603 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] PATH_INFO: `/v3/users/b5bba7d3b8824124ad900a8a20e499f7` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:31:02.658505 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5bba7d3b8824124ad900a8a20e499f7)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:31:02.660985 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:31:02.751786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5bba7d3b8824124ad900a8a20e499f7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:02.752457 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5bba7d3b8824124ad900a8a20e499f7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:02.769199 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5bba7d3b8824124ad900a8a20e499f7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:02.784270 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5bba7d3b8824124ad900a8a20e499f7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:02.785223 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5bba7d3b8824124ad900a8a20e499f7'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:02.824292 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1446/2904] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:31:02 2022] DELETE /identity/v3/users/b5bba7d3b8824124ad900a8a20e499f7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:31:02.828740 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a6f4-9819-45db-a686-b4456bc092ce None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:31:02.829252 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a6f4-9819-45db-a686-b4456bc092ce None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:31:02.829356 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8a6f4-9819-45db-a686-b4456bc092ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:31:02.874488 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-7bd8a6f4-9819-45db-a686-b4456bc092ce None None] MFA Rules not processed for user `c04d978bfed6492fb377cd274964302b`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:31:02.876142 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7bd8a6f4-9819-45db-a686-b4456bc092ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:03.026312 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1459/2905] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:31:03.039738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:31:03.041203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:03.140862 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f251abe-f603-4820-8205-398c9dcd3c32 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:31:03.148963 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39cb97-eed7-46b8-975f-95bcd4d770b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:31:03.149757 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39cb97-eed7-46b8-975f-95bcd4d770b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:31:03.150043 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39cb97-eed7-46b8-975f-95bcd4d770b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:31:03.150311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39cb97-eed7-46b8-975f-95bcd4d770b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:31:03.179115 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39cb97-eed7-46b8-975f-95bcd4d770b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:31:03.180954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39cb97-eed7-46b8-975f-95bcd4d770b8 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:31:03.187768 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:31:03.199552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1447/2906] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:31:03.800833 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a69a1-8ce4-4126-97ec-30ff4c14d3ae None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:31:03.802352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a69a1-8ce4-4126-97ec-30ff4c14d3ae None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:31:03.802352 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a69a1-8ce4-4126-97ec-30ff4c14d3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:31:03.804471 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8a6f4-9819-45db-a686-b4456bc092ce None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:31:03.805961 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7bd8a6f4-9819-45db-a686-b4456bc092ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:03.837494 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8a6f4-9819-45db-a686-b4456bc092ce None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:31:03.844472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:31:03.846391 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:31:03.846850 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:31:03.847348 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] PATH_INFO: `/v3/projects/512f3d98bc5e4481becbb60fcf9f1ea1` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:31:03.850052 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] RBAC: Authorizing `identity:delete_project(project_id=512f3d98bc5e4481becbb60fcf9f1ea1)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:31:03.852751 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:31:03.858738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-447a69a1-8ce4-4126-97ec-30ff4c14d3ae None None] MFA Rules not processed for user `7fd3a2891f734a659d52f308270bf6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:31:03.859954 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-447a69a1-8ce4-4126-97ec-30ff4c14d3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:03.944920 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512f3d98bc5e4481becbb60fcf9f1ea1', 'request_id': 'req-2f52333a-681f-47d8-a324-5d30b3e150af'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:03.960151 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512f3d98bc5e4481becbb60fcf9f1ea1', 'request_id': 'req-2f52333a-681f-47d8-a324-5d30b3e150af'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:03.976857 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512f3d98bc5e4481becbb60fcf9f1ea1', 'request_id': 'req-2f52333a-681f-47d8-a324-5d30b3e150af'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:03.993498 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] The token cache is being invalidate because project 512f3d98bc5e4481becbb60fcf9f1ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:31:03.993937 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:03.994864 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:31:03.997135 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1460/2907] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:31:03 2022] DELETE /identity/v3/projects/512f3d98bc5e4481becbb60fcf9f1ea1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:31:04.035734 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1448/2908] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:31:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:31:04.059225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:31:04.060933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:04.182808 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52333a-681f-47d8-a324-5d30b3e150af admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:31:04.189071 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:31:04.189847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:31:04.190218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:31:04.190639 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:31:04.193654 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:04.201766 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:31:04.272602 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:31:04.274806 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:31:04.284144 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1461/2909] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:31:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:31:14.010472 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=35 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:31:14.280253 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=24 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:31:19.547453 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e155143-0731-4f5d-a89c-4305da81922a None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:31:19.548014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e155143-0731-4f5d-a89c-4305da81922a None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:31:19.548014 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e155143-0731-4f5d-a89c-4305da81922a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:31:19.569642 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-6e155143-0731-4f5d-a89c-4305da81922a None None] MFA Rules not processed for user `7a5a70e8cad0452188dd1b750c4efa17`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:31:19.570852 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-6e155143-0731-4f5d-a89c-4305da81922a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:19.577719 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1449/2910] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:31:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:31:19.898258 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:31:19.899692 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:19.954827 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292c7cb-bab0-424a-8c27-b8dd659e254d service nova] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:31:19.959013 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:31:19.959553 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:31:19.959831 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:31:19.960105 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:31:19.961825 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:19.971215 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:31:19.973108 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:31:19.984686 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1462/2911] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:31:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:31:29.562729 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=3 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:31:29.977095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=26 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:31:36.072848 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e155143-0731-4f5d-a89c-4305da81922a None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:31:36.082542 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e155143-0731-4f5d-a89c-4305da81922a None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:31:36.086177 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfecba-9d13-42a7-bc01-52bcdd8201fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:31:36.086771 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfecba-9d13-42a7-bc01-52bcdd8201fa service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:31:36.087046 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfecba-9d13-42a7-bc01-52bcdd8201fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:31:36.087315 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfecba-9d13-42a7-bc01-52bcdd8201fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:31:36.088996 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-58cfecba-9d13-42a7-bc01-52bcdd8201fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:31:36.149727 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfecba-9d13-42a7-bc01-52bcdd8201fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:31:36.151148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfecba-9d13-42a7-bc01-52bcdd8201fa service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:31:36.160552 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1450/2912] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:31:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:31:46.156383 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=30 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:32:41.875767 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:32:41.878428 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:41.888998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ed2cf-9093-4754-a89c-ebb1d1ef10b9 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:32:41.892624 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b1856-a08e-465c-9aad-df82d963f26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:32:41.893218 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b1856-a08e-465c-9aad-df82d963f26b service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:32:41.893656 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b1856-a08e-465c-9aad-df82d963f26b service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:32:41.893925 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b1856-a08e-465c-9aad-df82d963f26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:32:41.895581 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-c65b1856-a08e-465c-9aad-df82d963f26b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:41.908642 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b1856-a08e-465c-9aad-df82d963f26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:32:41.910132 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b1856-a08e-465c-9aad-df82d963f26b service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:32:41.920810 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1463/2913] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:32:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:32:43.102860 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfecba-9d13-42a7-bc01-52bcdd8201fa service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:32:43.111326 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfecba-9d13-42a7-bc01-52bcdd8201fa service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:32:43.115467 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fa990-0bf9-468b-995a-d040f7e33a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:32:43.116074 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fa990-0bf9-468b-995a-d040f7e33a8d service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:32:43.116351 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fa990-0bf9-468b-995a-d040f7e33a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:32:43.116621 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fa990-0bf9-468b-995a-d040f7e33a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:32:43.118382 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-0f0fa990-0bf9-468b-995a-d040f7e33a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:43.132310 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fa990-0bf9-468b-995a-d040f7e33a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:32:43.133683 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fa990-0bf9-468b-995a-d040f7e33a8d service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:32:43.144375 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1451/2914] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:32:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:32:51.916022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=6 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:32:52.110618 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b1856-a08e-465c-9aad-df82d963f26b service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:32:52.120394 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b1856-a08e-465c-9aad-df82d963f26b service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:32:52.123981 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76a7d1-6c13-4c96-922e-40b67ad60b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:32:52.124573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a7d1-6c13-4c96-922e-40b67ad60b4a service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:32:52.124848 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a7d1-6c13-4c96-922e-40b67ad60b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:32:52.125117 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76a7d1-6c13-4c96-922e-40b67ad60b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:32:52.126913 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-4d76a7d1-6c13-4c96-922e-40b67ad60b4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:52.137286 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76a7d1-6c13-4c96-922e-40b67ad60b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:32:52.138803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76a7d1-6c13-4c96-922e-40b67ad60b4a service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:32:52.147217 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1464/2915] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:32:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:32:53.139608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=6 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:32:55.131991 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fa990-0bf9-468b-995a-d040f7e33a8d service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:32:55.141759 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fa990-0bf9-468b-995a-d040f7e33a8d service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:32:55.146002 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:32:55.146724 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:32:55.147003 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:32:55.147522 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:32:55.149398 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:55.159868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:32:55.161348 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:32:55.169787 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1452/2916] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:32:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:32:55.243249 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76a7d1-6c13-4c96-922e-40b67ad60b4a service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:32:55.251873 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76a7d1-6c13-4c96-922e-40b67ad60b4a service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:32:55.256095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:32:55.256695 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:32:55.256973 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:32:55.257247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:32:55.258986 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:55.272784 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:32:55.274247 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:32:55.285241 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1465/2917] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:32:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:32:55.346247 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:32:55.347614 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:55.414026 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839364d8-f3c3-4cf9-8503-ad61026c10a1 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:32:55.418786 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cf7df-db08-4867-a9ad-062f06e331f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:32:55.419376 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cf7df-db08-4867-a9ad-062f06e331f7 service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:32:55.419656 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cf7df-db08-4867-a9ad-062f06e331f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:32:55.419922 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cf7df-db08-4867-a9ad-062f06e331f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:32:55.421623 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-922cf7df-db08-4867-a9ad-062f06e331f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:55.433646 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cf7df-db08-4867-a9ad-062f06e331f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:32:55.435186 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cf7df-db08-4867-a9ad-062f06e331f7 service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:32:55.445890 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1453/2918] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:32:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:32:56.568614 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:32:56.570243 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:56.585899 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be55e9-abe8-4aeb-a0ce-127ad01fb026 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:32:56.592203 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea620b2-15ce-4929-9990-9c969180d5af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:32:56.592902 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea620b2-15ce-4929-9990-9c969180d5af service nova] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:32:56.593248 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea620b2-15ce-4929-9990-9c969180d5af service nova] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:32:56.593617 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea620b2-15ce-4929-9990-9c969180d5af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:32:56.595320 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-7ea620b2-15ce-4929-9990-9c969180d5af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:32:56.608856 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea620b2-15ce-4929-9990-9c969180d5af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:32:56.610508 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea620b2-15ce-4929-9990-9c969180d5af service nova] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:32:56.622352 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1466/2919] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:32:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:33:05.440640 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=38 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:33:06.617280 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=18 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:33:36.093240 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cf7df-db08-4867-a9ad-062f06e331f7 service placement] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:33:36.107369 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cf7df-db08-4867-a9ad-062f06e331f7 service placement] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:33:36.113581 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e85b16-1153-4b20-8367-d2ca5647f5a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:33:36.114203 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e85b16-1153-4b20-8367-d2ca5647f5a3 service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:33:36.114484 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e85b16-1153-4b20-8367-d2ca5647f5a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:33:36.114756 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e85b16-1153-4b20-8367-d2ca5647f5a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:33:36.117228 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f7e85b16-1153-4b20-8367-d2ca5647f5a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:33:36.132575 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e85b16-1153-4b20-8367-d2ca5647f5a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:33:36.134160 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e85b16-1153-4b20-8367-d2ca5647f5a3 service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:33:36.146819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1454/2920] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:33:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:33:36.973364 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ccd9-cc34-437a-8a39-ff74a33ed219 None None] REQUEST_METHOD: `POST` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:33:36.973910 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ccd9-cc34-437a-8a39-ff74a33ed219 None None] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:33:36.974026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ccd9-cc34-437a-8a39-ff74a33ed219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:33:36.994287 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.auth.core [None req-59c4ccd9-cc34-437a-8a39-ff74a33ed219 None None] MFA Rules not processed for user `7a5a70e8cad0452188dd1b750c4efa17`. Rule list: `[]` (Enabled: `True`). {{(pid=81338) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:33:36.995590 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-59c4ccd9-cc34-437a-8a39-ff74a33ed219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:33:37.002780 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1467/2921] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:33:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:33:46.141088 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=6 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:33:46.989293 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=3 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:33:55.355251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e85b16-1153-4b20-8367-d2ca5647f5a3 service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:33:55.367324 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e85b16-1153-4b20-8367-d2ca5647f5a3 service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:33:55.372706 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286f4e203e7a4a55bca0a58d061af2bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:33:55.373295 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:33:55.373596 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:33:55.373864 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:33:55.375543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:33:55.388107 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:33:55.389702 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:33:55.400159 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1455/2922] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:33:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:34:05.395604 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=6 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:34:29.283236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ccd9-cc34-437a-8a39-ff74a33ed219 None None] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:34:29.296064 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ccd9-cc34-437a-8a39-ff74a33ed219 None None] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:34:29.301130 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c210dc-1a81-4cdf-9883-4fc63236dc04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:34:29.301765 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c210dc-1a81-4cdf-9883-4fc63236dc04 service placement] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:34:29.302085 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c210dc-1a81-4cdf-9883-4fc63236dc04 service placement] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:34:29.302338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c210dc-1a81-4cdf-9883-4fc63236dc04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:34:29.304670 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-37c210dc-1a81-4cdf-9883-4fc63236dc04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:34:29.317561 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c210dc-1a81-4cdf-9883-4fc63236dc04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:34:29.318694 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c210dc-1a81-4cdf-9883-4fc63236dc04 service placement] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:34:29.331236 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1468/2923] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:34:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:34:32.423301 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:34:32.424935 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:34:32.489841 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40cf8b-9c5b-43c9-b263-f599060d086a service nova] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:34:32.493978 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:34:32.494597 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:34:32.494872 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:34:32.495137 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] PATH_INFO: `/v3/users/b7b64272dd774de5a7cd22776df18eca` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:34:32.496255 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7b64272dd774de5a7cd22776df18eca)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:34:32.498219 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:34:32.542796 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7b64272dd774de5a7cd22776df18eca'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.543441 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7b64272dd774de5a7cd22776df18eca'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.550304 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7b64272dd774de5a7cd22776df18eca'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.558171 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7b64272dd774de5a7cd22776df18eca'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.558792 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7b64272dd774de5a7cd22776df18eca'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.576741 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1456/2924] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:34:32 2022] DELETE /identity/v3/users/b7b64272dd774de5a7cd22776df18eca => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:34:32.744004 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c210dc-1a81-4cdf-9883-4fc63236dc04 service placement] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:34:32.758038 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c210dc-1a81-4cdf-9883-4fc63236dc04 service placement] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:34:32.819270 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:34:32.820070 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:34:32.820357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:34:32.820630 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] PATH_INFO: `/v3/projects/3277eaeb27724447a3864927021c2b79` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:34:32.821803 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3277eaeb27724447a3864927021c2b79)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:34:32.823739 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:34:32.878095 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3277eaeb27724447a3864927021c2b79', 'request_id': 'req-ede01531-1935-41b6-b66b-fc3563a8146b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.886454 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3277eaeb27724447a3864927021c2b79', 'request_id': 'req-ede01531-1935-41b6-b66b-fc3563a8146b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.893079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3277eaeb27724447a3864927021c2b79', 'request_id': 'req-ede01531-1935-41b6-b66b-fc3563a8146b'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.903437 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] The token cache is being invalidate because project 3277eaeb27724447a3864927021c2b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:34:32.903731 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.904333 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:32.905847 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1469/2925] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:34:32 2022] DELETE /identity/v3/projects/3277eaeb27724447a3864927021c2b79 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:34:32.909882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:34:32.911263 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:34:32.924627 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd345e-90d9-4326-bca9-bb5d647dcd00 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:34:32.981148 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:34:32.982069 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:34:32.982346 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:34:32.982748 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] PATH_INFO: `/v3/users/ad62e3b433094556b45ec3b2e13118a2` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:34:32.983907 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad62e3b433094556b45ec3b2e13118a2)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:34:32.985854 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:34:33.008543 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:34:33.037966 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad62e3b433094556b45ec3b2e13118a2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.038626 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad62e3b433094556b45ec3b2e13118a2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.045869 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad62e3b433094556b45ec3b2e13118a2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.054251 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad62e3b433094556b45ec3b2e13118a2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.054868 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-c5311bc5-0b28-434f-ab9f-42ce01e19b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad62e3b433094556b45ec3b2e13118a2'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.072335 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1457/2926] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:34:32 2022] DELETE /identity/v3/users/ad62e3b433094556b45ec3b2e13118a2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:34:33.243558 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:34:33.245175 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:34:33.258351 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede01531-1935-41b6-b66b-fc3563a8146b admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:34:33.313820 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:34:33.314448 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:34:33.314724 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:34:33.314998 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] PATH_INFO: `/v3/projects/5ccbddcb715647b684259719d1429890` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:34:33.316089 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ccbddcb715647b684259719d1429890)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:34:33.318034 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:34:33.339414 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:34:33.364782 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ccbddcb715647b684259719d1429890', 'request_id': 'req-a0b47f30-e043-4ef1-a13c-7793e455d5e5'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.371928 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ccbddcb715647b684259719d1429890', 'request_id': 'req-a0b47f30-e043-4ef1-a13c-7793e455d5e5'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.377972 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ccbddcb715647b684259719d1429890', 'request_id': 'req-a0b47f30-e043-4ef1-a13c-7793e455d5e5'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.388066 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] The token cache is being invalidate because project 5ccbddcb715647b684259719d1429890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:34:33.388350 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.388970 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:34:33.390472 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1470/2927] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:34:33 2022] DELETE /identity/v3/projects/5ccbddcb715647b684259719d1429890 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:34:43.068608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:34:43.385087 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:35:22.453772 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10192a2-6e1d-4670-8c0e-65e1472efb72 None None] REQUEST_METHOD: `POST` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:35:22.454316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10192a2-6e1d-4670-8c0e-65e1472efb72 None None] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:35:22.454316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10192a2-6e1d-4670-8c0e-65e1472efb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:35:22.473396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.auth.core [None req-f10192a2-6e1d-4670-8c0e-65e1472efb72 None None] MFA Rules not processed for user `7a5a70e8cad0452188dd1b750c4efa17`. Rule list: `[]` (Enabled: `True`). {{(pid=81337) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 20:35:22.474620 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f10192a2-6e1d-4670-8c0e-65e1472efb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:35:22.528273 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1458/2928] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 20:35:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 20:35:22.848246 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:35:22.849840 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:35:22.908804 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b47f30-e043-4ef1-a13c-7793e455d5e5 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:35:22.912342 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b599a5-2510-4b63-b8c0-cb316725a4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:35:22.912941 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b599a5-2510-4b63-b8c0-cb316725a4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:35:22.913213 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b599a5-2510-4b63-b8c0-cb316725a4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:35:22.913475 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b599a5-2510-4b63-b8c0-cb316725a4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:35:22.915179 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-50b599a5-2510-4b63-b8c0-cb316725a4a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:35:22.924501 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b599a5-2510-4b63-b8c0-cb316725a4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:35:22.925944 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b599a5-2510-4b63-b8c0-cb316725a4a0 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:35:22.934262 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1471/2929] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:35:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:35:32.521237 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=25 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:35:32.930187 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=26 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:36:10.879765 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10192a2-6e1d-4670-8c0e-65e1472efb72 None None] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:36:10.881188 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-f10192a2-6e1d-4670-8c0e-65e1472efb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:36:10.951245 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10192a2-6e1d-4670-8c0e-65e1472efb72 None None] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:36:10.956108 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:36:10.956711 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:36:10.956988 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:36:10.957285 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:36:10.959007 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:36:10.971388 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:36:10.972957 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:36:10.983523 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1459/2930] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:36:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:36:20.978663 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=32 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:37:14.496207 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b599a5-2510-4b63-b8c0-cb316725a4a0 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:37:14.504791 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b599a5-2510-4b63-b8c0-cb316725a4a0 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:37:14.508317 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:37:14.509022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:37:14.509302 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:37:14.509598 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:37:14.511267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:37:14.521551 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:37:14.522959 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:37:14.531504 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1472/2931] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:37:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:37:24.527606 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=6 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:37:28.812316 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:37:28.813751 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:37:28.825426 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d77e-05ba-4b69-a927-75956888d9f5 service placement] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:37:28.828993 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f5a88f263014f15a6f59cfdaa2bb6d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:37:28.829606 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:37:28.829882 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:37:28.830152 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:37:28.831819 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:37:28.895167 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:37:28.896578 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:37:28.906535 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1460/2932] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:37:34.749118 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:37:34.751267 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:37:34.766509 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07303cb9-53f0-4d6d-9fef-cbf0d4999d65 service neutron] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:37:34.771433 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50d371-afc7-47c4-b3da-df8c571a6079 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:37:34.772079 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50d371-afc7-47c4-b3da-df8c571a6079 service placement] REQUEST_METHOD: `GET` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:37:34.772357 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50d371-afc7-47c4-b3da-df8c571a6079 service placement] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:37:34.772625 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50d371-afc7-47c4-b3da-df8c571a6079 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:37:34.774338 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-cd50d371-afc7-47c4-b3da-df8c571a6079 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:37:34.786346 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50d371-afc7-47c4-b3da-df8c571a6079 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:37:34.787905 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50d371-afc7-47c4-b3da-df8c571a6079 service placement] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:37:34.798814 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1473/2933] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:37:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:37:34.814743 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:37:34.816317 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:37:34.830291 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b979a-33d8-403a-bcfc-16a169fafc41 service neutron] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:37:34.835158 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:37:34.835780 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:37:34.836057 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:37:34.836328 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] PATH_INFO: `/v3/users/7a5a70e8cad0452188dd1b750c4efa17` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:37:34.837443 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5a70e8cad0452188dd1b750c4efa17)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:37:34.839396 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:37:34.886747 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5a70e8cad0452188dd1b750c4efa17'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:34.887959 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5a70e8cad0452188dd1b750c4efa17'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:34.894980 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5a70e8cad0452188dd1b750c4efa17'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:34.902519 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5a70e8cad0452188dd1b750c4efa17'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:34.903804 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5a70e8cad0452188dd1b750c4efa17'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:34.920994 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1461/2934] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:37:34 2022] DELETE /identity/v3/users/7a5a70e8cad0452188dd1b750c4efa17 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:37:35.100088 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50d371-afc7-47c4-b3da-df8c571a6079 service placement] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:37:35.113022 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50d371-afc7-47c4-b3da-df8c571a6079 service placement] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:37:35.169445 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:37:35.170101 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:37:35.170371 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:37:35.170634 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] PATH_INFO: `/v3/projects/3dae2633ff47484981df67d10b94de47` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:37:35.171778 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dae2633ff47484981df67d10b94de47)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:37:35.173658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:37:35.220128 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dae2633ff47484981df67d10b94de47', 'request_id': 'req-84db793b-457e-4bb7-839d-9b1551aab481'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.227229 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dae2633ff47484981df67d10b94de47', 'request_id': 'req-84db793b-457e-4bb7-839d-9b1551aab481'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.233889 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dae2633ff47484981df67d10b94de47', 'request_id': 'req-84db793b-457e-4bb7-839d-9b1551aab481'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.243762 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] The token cache is being invalidate because project 3dae2633ff47484981df67d10b94de47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:37:35.244043 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.244658 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.246148 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1474/2935] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:37:35 2022] DELETE /identity/v3/projects/3dae2633ff47484981df67d10b94de47 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:37:35.250607 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:37:35.251999 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:37:35.264818 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ff040-e044-46f8-bcde-88e332e78f04 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:37:35.319122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:37:35.319675 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:37:35.319950 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:37:35.320221 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] PATH_INFO: `/v3/users/7fd3a2891f734a659d52f308270bf6ce` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:37:35.321608 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd3a2891f734a659d52f308270bf6ce)` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:37:35.323492 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:37:35.328613 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:37:35.370061 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd3a2891f734a659d52f308270bf6ce'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.370715 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd3a2891f734a659d52f308270bf6ce'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.377738 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd3a2891f734a659d52f308270bf6ce'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.384609 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd3a2891f734a659d52f308270bf6ce'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.385238 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.notifications [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd3a2891f734a659d52f308270bf6ce'} {{(pid=81337) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.402337 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1462/2936] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 20:37:35 2022] DELETE /identity/v3/users/7fd3a2891f734a659d52f308270bf6ce => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 20:37:35.573443 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] Authenticating user token {{(pid=81338) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:37:35.574900 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.fernet_utils [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81338) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:37:35.587225 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db793b-457e-4bb7-839d-9b1551aab481 admin admin] Validating token access rules against request {{(pid=81338) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:37:35.649745 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c04d978bfed6492fb377cd274964302b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e6d76b1c203e49b8ae5f7f394a20a597', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81338) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:37:35.650573 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:37:35.650851 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] SCRIPT_NAME: `/identity` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:37:35.651119 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] PATH_INFO: `/v3/projects/9b9a9e5e833c4d19bd06606517350d22` {{(pid=81338) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:37:35.652219 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9a9e5e833c4d19bd06606517350d22)` {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:37:35.655726 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] RBAC: Authorization granted {{(pid=81338) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:37:35.676156 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:37:35.702776 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9a9e5e833c4d19bd06606517350d22', 'request_id': 'req-75df6ae0-a484-47a8-a0e0-af9f24ef423d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.710111 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9a9e5e833c4d19bd06606517350d22', 'request_id': 'req-75df6ae0-a484-47a8-a0e0-af9f24ef423d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.716933 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9a9e5e833c4d19bd06606517350d22', 'request_id': 'req-75df6ae0-a484-47a8-a0e0-af9f24ef423d'} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.727026 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] The token cache is being invalidate because project 9b9a9e5e833c4d19bd06606517350d22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81338) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 20:37:35.727319 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.727946 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG keystone.notifications [None req-75df6ae0-a484-47a8-a0e0-af9f24ef423d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81338) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 20:37:35.729407 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: [pid: 81338|app: 0|req: 1475/2937] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 20:37:35 2022] DELETE /identity/v3/projects/9b9a9e5e833c4d19bd06606517350d22 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 20:37:45.398908 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:37:45.724084 dv-cinder-801522-12-iscsi devstack@keystone.service[81338]: DEBUG dbcounter [-] [81338] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81338) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 20:41:14.818572 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] Authenticating user token {{(pid=81337) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 20:41:14.820133 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:41:14.956767 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede52042-d5c6-4ddf-b8d7-5e3e211c3e37 admin admin] Validating token access rules against request {{(pid=81337) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 20:41:15.013340 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c3c7d-ed96-4dcd-ac39-881bdd9a240a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5181c4b31ac240469aff7a0aee35eb7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3130efb60669479aae97cf2ec518e76d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81337) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 20:41:15.014036 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c3c7d-ed96-4dcd-ac39-881bdd9a240a service placement] REQUEST_METHOD: `GET` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 20:41:15.014311 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c3c7d-ed96-4dcd-ac39-881bdd9a240a service placement] SCRIPT_NAME: `/identity` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 20:41:15.014576 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c3c7d-ed96-4dcd-ac39-881bdd9a240a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81337) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 20:41:15.016265 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.fernet_utils [None req-7b8c3c7d-ed96-4dcd-ac39-881bdd9a240a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81337) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 20:41:15.027122 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c3c7d-ed96-4dcd-ac39-881bdd9a240a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 20:41:15.028669 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c3c7d-ed96-4dcd-ac39-881bdd9a240a service placement] RBAC: Authorization granted {{(pid=81337) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 20:41:15.038879 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: [pid: 81337|app: 0|req: 1463/2938] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 20:41:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 20:41:25.034730 dv-cinder-801522-12-iscsi devstack@keystone.service[81337]: DEBUG dbcounter [-] [81337] Writing DB stats keystone:SELECT=89 {{(pid=81337) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}